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(57) Abstract 



An interactive dental restoration 
method, a system for use between a 
dentist, and a dental restoration lab- 
oratory. The method includes iden- 
tifying a dental restoration need in a 
patient; designing a preliminary treat- 
ment plan that includes design criteria 
for preparation of a dental prosthesis 
to be placed in the patient to satisfy 
the dental restoration need; transmit- 
ting the preliminary treatment plan via 
a communications network to a den- 
tal restoration laboratory; and com- 
municating a final treatment plan, in- 
cluding modifications to the prelimi- 
nary treatment plan where necessary, 
to the dentist. The 'system includes 
a computer-based dental restoration 
system of a network server (1610) 
having a database (1630), a network 
(1620), and at least one local computer 
(14). The database stores informa- 
tion about materials, procedures, and 
preparations concerning dental pros- 
theses. 
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INTERACTIVE DENTAL RESTORATIVE NETWORK 

TECHNICAL FIELD 

The invention is directed to methods, systems and devices for dental restoration 

5 wherein communication between the dentist and restoration laboratory are held in real time 
to discuss, finalize and optimize a treatment plan for a patient. More specifically, the 
invention is directed to an interactive computer-based system and method to enable the 
dentist and restoration laboratory to analyze color images of one or more teeth and teeth 
preparation so that a replacement tooth or crown can be particularly designed to precisely 

10 match the tooth that is to be replaced in certain clinical or cosmetic procedures. 

BACKGROUND OF THE INVENTION 

Restorative dentistry is the art and science of replacing or restoring lost tooth 
structure. The amount of tooth structure to be replaced determines what path the operator 

15 takes - whether the restoration will be a crown, bridge, inlay, onlay or direct restoration 
(i.e., a filling). The choice of that path in the past was more simple, due to the limited 
number of materials and techniques available. For example, U.S. Patents 5,766,006 and 
5,961,324 describe methods and systems for determining tooth color information based 
upon digital images provided by a camera and then matching the color of the restoration 

20 article (i.e., dental prosthesis) with the determined tooth color. In recent years, however, 
with the advent of new materials and concepts,; treatment choices have expanded in a 
phenomenal way. Dentists are now facing an overload of information in trying to decide 
which materials and procedures are the best suited for their particular cases. What the state- 
of-the-art practitioner needs is a source to be able to go to, at a moment's notice, that will be 

25 able to aid him and his lab if necessary in treatment planning and delivering the best 

restorative dentistry possible, utilizing the most appropriate materials available today. The 
present invention now satisfies this need. 

SUMMARY OF THE INVENTION 

30 The invention relates to an interactive dental restoration method between a dentist 

and a dental restoration laboratory. The basic steps of this method include identifying a 
dental restoration need in a patient; designing a preliminary treatment plan that includes 
design criteria for preparation of a dental prosthesis to be placed in the patient to satisfy the 
dental restoration need; transmitting the preliminary treatment plan via a communications 

35 network to a dental restoration laboratory; and communicating a final treatment plan, 
including modifications to the preliminary treatment plan where necessary, to the dentist. 
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Typically, the final treatment plan includes information about materials for preparing a 
dental prosthesis that satisfies the design criteria, and the dental prosthesis is then prepared 
for placement in the patient. This method enables optimization of the dental restoration 
with significant savings in time and effort for the dentist, dental technician and the patient. 

5 Generally, the dentist prepares the preliminary treatment plan and the design criteria 

include digital image representations of ;the dental restoration need. Thereafter, the 
preliminary treatment plan can be forwarded to and evaluated by the laboratory before a 
final treatment plan is formulated and communicated to the dentist. The step of transmitting 
and evaluating the plan are codirected over the communications network. Thus, the final 

10 treatment plan is not implemented in the patient until after interim preparation information 
is transmitted to the laboratory and confirmed, thus avoiding rework or revision after the 
plan has been implemented. 

Advantageously, the design criteria or the modifications thereto include proposed 
decay excavation, tooth preparation, or dental prosthesis color. When a dental prosthesis 

15 such as a crown, bridge or replacement tooth is needed, the method includes verifying that 
the dental prosthesis is prepared according to the final treatment plan prior to placement of 
the dental prosthesis in the patient. In order to obtain the best color match of the dental 
prosthesis with the patient's teeth, the digital image representations include REAL IMAGE 
and REFERENCE IMAGES and the modifications include correlation of a color selection 

20 for the dental prosthesis to match the REAL IMAGE. Furthermore, the design criteria can 
include tooth preparation and proposed decay excavation, and the method further comprises 
a communication of a confirmation or modification, from the laboratory, of die acceptability 
of one or more of the proposed design criteria. 

The invention also relates to a computer-based dental restoration system comprising 

25 a network server having a database storing information about materials, procedures and 
preparations concerning dental restoration prosthesis; a communications network providing 
access to the network server; and one or more computers at a dental office accessing 
information stored at the database over the communications network and displaying the 
information in a humanly readable format. Preferably, the communications network is the 

30 Internet, and the information stored in the database comprises preparation diagrams, 
reduction dimensions, margin design and burs for specific dental restoration prostheses. 

Advantageously, the database further stores information concerning one or more 
patients having dental restoration needs. Also, the network server further comprises 
application programs for enabling users to query the database regarding specific materials 

35 or procedures concerning dental restoration prostheses for confirmation, verification, 

modification or evaluation of the same, with the one or more computers at the dental office 
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receiving answers from the database to such queries. If desired, a printer located at the 
dental office can be used to print these answers for use by the dentist in carrying out the 
treatment plan. 

The dental restoration laboratory also includes at least one computer that has access 
to the network server and the computer(s) at the dental office over the communications 
network. Preferably, the system includes a digital camera for taking digital images of the 
patient's teeth that are in need of dental restoration and a communication link for 
transmitting the digital images to the computer(s) at the dental office. Also, the 
computer(s) at the dental office store these digital images and the communications network 
forwards the digital images to the database for storage therein. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other aspects of the invention should be more apparent from the 
following detailed description and drawings in which: 

FIG. 1 illustrates a shade analyzer system for capturing images in accord with a 
specific embodiment of this invention; 

FIG. 2 shows a representative image captured by the system of FIG. 1; 

FIG. 3 shows a representative image made up of pixels as captured by detector 
elements of the system in accord with the invention; 

FIG. 4 illustrates processing of the image of FIG. 3 using pseudo-pixels, in accord 
with one preferred embodiment of the invention; . . . 

FIG. 5 shows an end piece constructed for use with the system in FIG. 1, for 
simultaneous processing of an actual tooth image and various reference tooth shades; 

FIG. 6 illustrates a compression sleeve constructed according with a specific 
embodiment of the invention for capturing high-quality tooth images; 

FIG. 7 illustrates a source-to-tooth illumination, for improved image capturing in 
accord with one embodiment of the invention; 

FIG. 8 illustrates baffling and stray-light rejection within a sleeve in a specific 
embodiment of the invention; 

FIG. 9 illustrates different pseudo-pixel imaging mechanisms, optionally dependent 
upon tooth shape characteristics, used in accord with the invention; 

FIG. 10 illustrates a non-contact tooth imaging system, with stray light rejection, 
constructed according to a specific embodiment the invention; 

FIG. 1 1 illustrates another embodiment of a non-contact tooth imaging system in 
accordance with the present invention; 

FIG. 12 illustrates a digital image of a tooth; 
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FIG. 13 illustrates a system for reimaging a tooth; 

FIG. 14 illustrates various tooth decay patterns and restorations that can be 
addressed in accordance with the present invention; 

FIG. 15 is a schematic representation of the interactive network system of the 
5 invention. 

FIG. 16 is a block-diagram of the configuration of the interactive network system of 
the invention. 

DETAILED DESCRIPTION OF THE INVENTION 

10 The present invention now provides an enhanced dental restoration network as a 

service for dentists. This network would be established via a computerized link between the 
dentist, the lab, and, optionally, the lab's databank of the most current information 
regarding materials, procedures, and other services such as preparation design and 
surveying for dental restoration prostheses such as caps, crowns, bridges, fillings and the 

15 like. 

In a typical case, initial steps of complete examination and diagnosis of the patient's 
dental condition is by the dentist. This generally includes a basic periodontal examination, 
clinical exam, radiographs, screening for TMD; etc.. The dentist also creates a preliminary 
treatment plan for addressing the dental neeiis of the patient. When tooth capping or 

20 replacement is required, clinical pictures which can are taken and captured on a program 
and are forwarded to the lab. Theses pictures can be of the color of the patient's teeth, the 
preparation of a tooth for further treatment, or even of a temporary treatment which can be 
modified or enhanced before being finalized. The pictures can be taken in any one of a 
number of ways, as described in more detail below. 

25 In this invention, an on-site advanced restorative system is provided where the 

dentist takes one or more digital images of the tooth prior to restoration, eliminates areas of 
decay in the image, and matches the shade of material to be used to restore the tooth based 
upon the digital images of the tooth prior to removal. In another aspect, the dentist takes 
digital images of the tooth after preparation and matches the shade of the material to be used 

30 in restoration based upon the remaining parts of tooth. These pictures can be forwarded by 
facsimile, direct computer link, or by e-mail to the lab for evaluation, along with the 
dentist's preliminary treatment plan. 

After preliminary treatment plans are designed, and areas such as periodontal needs, 
decay excavation, endodontic concerns are addressed, the restorative needs are considered. 

35 If the treatment plan may include fixed prosthodontics (crowns and bridges), the clinical 
pictures are then forwarded to the lab. The doctor and. technician assess the case together 
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prior to accessing the interactive dental restorative network ("the site"). An illustration of 
such a network is shown in Fig. 16 and described in more detail below. If the case involves 
only direct restorations, the dentist can go directly to the site. 

If the dentist does not have access to the site but his lab does, the doctor could send 
5 pictures to the lab, the technician in turn could access the site, and consult back to the 

doctor giving restorative options as given by the site. This service would be rendered by the 
labs for dentists that are not comfortable or familiar enough with computers to do their own 
electronic processing and communication with the site. 

The site would provide access for users to information on materials, procedures 
10 involved in using such materials such as preparation design, recommended burs to achieve 
such a preparation, recommended temporization materials, cements that should be used with 
that given material, instructions on how to use such a cement (i.e., conditions such as 
whether one should etch or prime, for how long, whether to dry it or not, to pre-cure it or 
not, etc.), and where to buy such materials. Alternatively, the lab could explain how it 
15 would provide such a service or who the dentist could contact to obtain such service. 
Beyond this, once the treatment is underway, the dentist could verify his preparation with 
the technician, if necessary, by sending digital images electronically for review prior to final 
impressions. For a more precise evaluation of the case in treatment, the dentist would scan 
his preparations and go to part of the site that would survey the teeth and assess reduction 
20 amount. This would typically be used in larger and more complex cases. 

The site would offer a number of features for communicating dental restoration data 
to the dentist and technician. One of the most unique features of the site is that it is 
interactive. Rather than just being a databank of information for the dentist to review, the 
dentist would be led through a step by step procedure to determine the most appropriate 
25 restorative path to take. The site could be visited periodically to consider alternative 

procedures, different options or to just confirm that the previous recommendations are clear 
and are being followed. Although many dentists read articles and reports, and attend 
seminars to obtain the latest information, until there is a case in hand, a lot of that 
information is not applicable. By the time a given case corresponds to a case presented at a 
30 previous seminar, the dentist may have already forgotten the information. The present 
method and system provides immediate feedback of the most up-to-date information in real 
time for the specific need of the current patient. 

As the dentist accesses the site, he can immediately be asked certain questions 
regarding the patient's history. Typical questions under consideration for dental restoration 
35 procedures include: Are esthetics a main concern? Is the patient a bruxer (i.e., heavy 

grinder)? What is the extension of the patient's smile - give the teeth numbers of the limits 
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of what is visible on their widest smile? Does the patient have a high lip line, i.e., does their 
lip lie below the incisal edge, midtooth, at the cervical margin, above the cervical margin? 
Do they show mandibular (lower) teeth when they smile? Is the opposing occlusion 
natural? If not, is it metal, porcelain, amalgam, composite or denture teeth? By providing 

5 the restoration lab with this additional information, all factors can be considered so that a 
sound, tailored treatment plan can be confirmed or recommended. 

The laboratory then would consider the teeth in question: Are they anterior or 
posterior? Endodontically treated or vital? What shade are they initially? What is the. 
desired shade? What is the prepared tooth shade? (Information on how to obtain the color 

1 0 of the teeth is disclosed in the other application- - please insert here) What are the 

dimensions of the tooth - is it a short clinical crown, or average to larger than average in 
size? Are there any implants involved? The process would operate like an "elimination 
tree" - if the firstquestion of esthetic concern was a "no", the site would not go on to ask 
smile dimensions and such. All questions would be answered to a point to compile a 

15 profile, and any given patient may require their case to be divided into more than one 

profile depending on the scope of their needs, say corresponding to sections of their mouth 
in quadrants. 

Another issue to be addressed is that of materials. This would give the material 
name, its characteristics and properties, and why it was suggested. Suggested is the 

20 operative word here because it should be understood that ultimately it is the dentist's choice 
of treatment modality that is used and not that of the technician or the site. After the dentist 
chooses the material, he would need to know where to obtain it, if he didn't already have 
access to it. This would entail the dentist purchasing a direct material, either through the 
site to an ordering area, or being referred to a lab in his area that uses such a system. 

25 The issue of preparation design when planning to use that given material is also 

addressed by the site. Different materials demand different substructures and margins. 
There are not a tremendous number of different designs needed. Within the site, there is a 
file of preparation diagrams, which could be printed out by the dentist, if necessary, to 
provide reduction dimensions, margin design and the burs needed to do this. This would 

30 include bur name and number, type and where to obtain them. Once again, the dentist may 
order this through the site or obtain information or where it could be purchased in his area. 
The dentist could simply obtain all materials from the site compile a shopping list for this 
particular procedure and go out and obtain the materials on his own. 

Once the case is underway, and the initial preparations are completed, the dentist 

35 could go back to the site and scan the preparations to assure compliance. Alternatively, 
digital representations of the preparations could be sent back to the site or lab for their 
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further review. A of the preparations can also be made by accessing a survey area of the 
site. This would evaluate the preparation for undercuts, under-reduction, margin extension, 
and highlight areas that needed to be modified for optimum results. 

Communication with the site in real time would save a great amount of time and 

5 effort. By first confirming that the preparations and recommended dental restoration 
procedures are correct, the lab would not have to be pouring and working on models of a 
case that were not useable because the preparations required changing. Also saved would 
be the time of having the patient return to the office on multiple occasions for refining 
preparations. This is a significant benefit for both dentist, patient and laboratory. By 

10 providing the laboratory operator with such information without taking an impression, 

pouring up models and surveying them with a traditional surveyor, both time, materials and 
expenses due to re- work are saved. 

Another advantage of real time evaluation is reduction. One of the most common 
errors in preparation is under-reduction (i,e ; , not, removing enough tooth structure to allow 

15 room for the materials that will make up the crown or restoration), which causes either too 
thin of restoration in that area which can lead to future failure, or repreparation (i.e., more 
wasted chair time) and new impressions or reduction copings. Within the survey site, the 
dentist would be able to more accurately scan the preparation with the teeth in occlusion so 
as to measure the amount of reduction to the tenth of a millimeter. Then, the dentist would 

20 compare this measurement to the given specifications of the preparation he had retrieved 
earlier from the preparation design area of the site to confirm compliance. 

Another use of the present interactive dental restoration network is the verification 
of the final dental prosthesis before it is permanently placed in the patient's mouth. For 
example, when a crown is finally prepared, digital images can be taken and compared to the 

25 digital images of the patient's teeth that were previously obtained to assure that the closest 
color match has been achieved. Any necessary color corrections can be made by the 
laboratory or the technician before permanent placement of the crown. This again saves 
time by avoiding rework when the patient returns to the dentist's office for the installation 
of the crown. 

30 A number of different aspects of determining tooth shade color are disclosed. For 

clarity of presentation, these aspects are organized and described below in sections which 
are not intended to be limiting in any way. 
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The Color Determination Method 

With reference to Fig. 1, a solid state camera 12 (e.g., a CCD camera coupled to a 
PC board, or an intra-oral camera) is utilized to capture one or more images of each known 
conventional tooth shade. Tooth shades used to this end may correspond, for example, to 

5 the VITA™ Shade guide, or a shade guide corresponding to a porcelain by another dental 
products manufacturer. By way of example, a first series' of images taken in accordance 
with the invention corresponds to sequential images of the Al shade by Vita, a second series 
of images corresponds to sequential images of the A2 shade by Vita, and so on. In 
accordance with the invention, captured image series of the known 7 tooth shade guides are 

10 properly labeled and then stored onto the hard disk of the computer, or other suitable 
storage device for further analysis. FIGr 2 illustrates a representative digital image 30 
captured by the camera 12. 

As known in the art, each digital image has a plurality of picture elements, i.e., 
"pixels", corresponding to the elements of the solid state camera and representing the light 

15 intensity and color properties of a given spatial location on the tooth. The distance between 
adjacent pixels in the image is determined by the spatial resolution of the camera. For 
example, an image of a tooth shade (or a human tooth) can be made up of 300 pixels in 
width across the tooth and 350 pixels in height. In human teeth, any given tooth is 
approximately the same size, give or take a couple of millimeters, for all people. For 

20 example, most central incisors. usually measure between 9-11 mm in width, and somewhat 
greater in length It is clear therefore that for a given spatial resolution of the camera, in 
accordance with this invention an image of a tooth can be taken knowing the approximate 
number of pixels corresponding to the tooth in the image. Thus, in the example above, 
1 mm of the tooth width may be represented by 30 pixels. It will naturally be appreciated 

25 that the tooth image is typically not rectangular, and that pixels at the comers of an image 
may correspond to the background (i.e., the region outside of the tooth) and not of the tooth 
or tooth shade. See FIG. 2 for further illustration. 

As indicated above, a single digital image can be captured for each tooth shade or 
actual tooth. Those skilled in the art will appreciate, however, that taking a series of images 

30 per shade is preferable, since it reduces the risk of image anomalies, as explained in further 
detail below. 

Next, each image or each image in a series is processed into a "pseudo" reference 
image (hereinafter PSEUDO IMAGE) made up of pseudo-pixels. As used in this 
disclosure, pseudo-pixels correspond to groups of pixels covering specific areas (i.e., 
35 rectangular or square) of the image plane. FIG. 3 shows a blow up image of the pixels 42 
(only representative pixels 42 are shown) which make up the tooth image 40 of FIG. 2. In 
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accord with the invention, these pixels are transformed into pseudo-pixels 44, as shown in 
FIG. 4. In the embodiment illustrated in FIG. 4 each pseudo-pixel 44 is made up of all real 
pixels within the area of the associated pseudo-pixel. Pseudo-pixels 44b shown in the 
figure illustrate, for example, how a pseudo-pixel can be generated from nine real pixels 42. 

5 FIG. 4 also illustrates that an image can be made from either a real tooth 40 (resulting in a 
REAL IMAGE) or a reference shade 40 (resulting in a REFERENCE IMAGE). 

FIG. 9 shows how pseudo-pixels can be; arranged in a preferred embodiment in 
different patterns, automatically, depending upon which tooth is imaged. For example, an 
incisor 200 can have an arrangement of pseudo-pixels 202, as shown in the left-hand 

10 example, while a molar 204 can have an arrangement of pseudo-pixels 206, as shown in the 
right-hand example. With further reference to FIG. 1, such arrangements can be made 
automatically in, the system of this invention by informing the computer 14, and hence the 
software 50, to apply pseudo-pixels in the appropriate pattern. Such arrangements assist in 
overall processing by ensuring appropriate pseudo-pixel placement. As illustrated in FIG. 

1 5 9, pseudo-pixels need not be contiguous, or aligned, such as shown by the arrangement of 
pseudo-pixels 206. 

In a preferred embodiment, the intensity and color associated with a pseudo-pixel 
are computed or otherwise formed as an average (or other statistical measure) of the actual 
pixels forming the pseudo-pixel By way of example, if an actual image taken in the first 

20 step of the method corresponds to a rectangular tooth that is digitized at 300W by 350H 
resolution, i.e., having a total of 300x350 elements, in accordance with this embodiment 
one can create pseudo-pixels such as 6W by 7H, with each pseudo-pixel being formed as a 
statistical measure of the 50x50 pixels within the pseudo-pixel (resulting in 42 pseudo- 
pixels representing the entire tooth). 

25 As noted above, in accordance with a preferred embodiment, pseudo-pixels are 

generated by data derived from the actual pixels located within the pseudo-pixel. For 
example, in a specific embodiment one can average the red, green and blue (RGB) 
components for each of the 2500 pixels within each pseudo-pixel to determine a reference 
RGB for that pseudo-pixel. Those skilled in the art will appreciate that other statistical 

30 measures or characteristics can be used, such as the mean "hue" measure of the pixels 

within a pseudo-pixel, or others. For example, the RGB pixel values may be converted into 
the Hue, Saturation and Intensity ("HSI") color space by using known algorithms, such as 
the Gonzalez and Woods method,, as follows: 

35 
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R = Red value for pixel 
G = Green value for pixel 
B = Blue value for pixel 
Intensity = 1/3 (R+ G + B) 
5 1 Saturation = 1 - (3/ R +G + B)). * Min(R, G, B) 

Hue = Cos' 1 ( (0.5* ((R -G) + (R^B))) A ((R -B) * (G - B)) °* 5 ) 

If S = 0, Hue is meaningless . 

If (B/Intensity) > (G/Intensity) then Hue = 360 - Hue 

Since Hue is an angle in degrees values were normalized to 0.1 with Hue = Hue/360 
10 As known in the art, the RGB color space can be represented as a simple cube, with 

R, G and B emanating from one corner along three perpendicular edges. The origin corner 
(0,0,0) is black, and the opposite corner (1,1,1) is white. All points along this line from 
corner to corner are shades of grey. The HSI color space is this same cube stood on the 
origin corner, with the Black White line being vertical. The black - white line is the 
15 intensify axis, the hue is given by an angle from the intensity axis and the saturation is the 
distance from the intensity axis to the color point (i.e., the radius). The new VITAPAN 3D- 
Master Shade system uses an L*a*b* Color Sphere to determine tooth shades based on 
Value, Chroma and Hue. It is possible to convert the RGB values to this color system, if 
necessary or desired. 

20 In accordance with a preferred embodiment, PSEUDO IMAGES corresponding to 

series of images are processed then into a "REFERENCE IMAGE". By way of example, 
the REFERENCE IMAGE is generated as an average (or other statistical measure) of the 
PSEUDO IMAGE series of images of the VITA™ A2 shade guide. The measure in this 
example is obtained by averaging the R, G, B components for each pseudo-pixel of the A2 

25 PSEUDO IMAGE series to determine an ?ty prage RGB value for a pseudo-pixel of the 
REFERENCE IMAGE. In alternative embodiments operating in the HSI color space, the 
corresponding average values can also be determined for each pseudo-pixel of the PSEUDO 
IMAGE series; and, for example, an average hue (or other statistical measure of hue) can 
also be associated with each pseudo-pixel in the REFERENCE IMAGE. Those skilled in 

30 the art will appreciate that other color characteristics can be used alternatively or in 

conjunction with the measure of RGB and/or hue. It will be appreciated that if only one 
PSEUDO IMAGE is made per shade, than that PSEUDO IMAGE defaults as the 
REFERENCE IMAGE since no other statistical combination is available. 

It should be noted that forming of pseudo-pixels is not a requirement for practicing 

35 this invention. However, pseudo-pixels are used in a preferred embodiment because they - 
may reduce the processing load of the system, minimize storage requirements and also 
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because they can simplify the task of aligning corresponding pixels from different images. 
Proper pixel alignment is important in order to ensure the integrity and accuracy of the 
statistical averages used in the formation of REFERENCE IMAGES. In this regard it will 
be appreciated that it is generally difficult to precisely align all pixels in several images , 

5 taken from the shades of the same shade guide, unless there is extremely good control 
utilized in the image capture sequence. Using pseudo-pixels in accordance with the 
preferred embodiment reduces the total number of pixels per image and thus simplifies the 
task of aligning different images accurately. 

Pseudo-pixels , are even more important in later steps of the processing method of 

10 this invention. That is, although one has complete freedom to set up the optics and the 
camera, which together determine the magnification of a captured tooth shade (or tooth) 
image, when trying to "match" a REFERENCE IMAGE to an actual digital image of a 
patient's tooth, the actual image (hereinafter referred to as a SNAPSHOT) may be quite 
different in shape and size (either real size or apparent size due to magnification differences 

1 5 in the optics or camera CCD element size). As such, a "one-to-one" comparison between 
the SNAPSHOT and the REFERENCE IMAGE is difficult. Pseudo-pixels help in this 
respect because the SNAPSHOT can be scaled to approximate the REFERENCE IMAGE 
size, or vice versa; and the SNAPSHOT can also be processed into pseudo-pixels. The 
scaled and pseudo-pixel version of the SNAPSHOT image is denoted as the "REAL 

20 IMAGE" hereinafter. Pseudo-pixels used in a preferred embodiment thus permit a 
convenient mechanism for comparing a REFERENCE IMAGE to a REAL IMAGE. 

In accordance with a preferred embodiment, the generation of each REFERENCE 
IMAGE preferably includes a "bad pixel" routine where each pseudo-pixel in the PSEUDO 
IMAGE series is analyzed for bad pixels. A "bad pixel" means any real pixel 

25 corresponding to a defective CCD element or corresponding to an area with an unwanted 
artifact, e.g., reflection, in the digital image, or an area that contains "background" imagery 
(e.g., any pixel image not corresponding to the tooth or tooth shade). Any pseudo-pixel in 
the PSEUDO IMAGE which contains a bad pixel is preferably not utilized in the generation 
of the REFERENCE IMAGE. That is, if for example a REFERENCE IMAGE is made up 

30 as an average of three PSEUDO IMAGES, and yet one pseudo-pixel in one of the PSEUDO 
IMAGES contains a bad pixel, then in a specific embodiment the resulting pseudo-pixel of 
the REFERENCE IMAGE is either discarded, or computed only as an average of the other 
two associated pseudo-pixels of. the PSEUDO IMAGES. 

Note that the bad pixel routines are particularly important at the edges of the image 

35 of the tooth or tooth shade. Consider, for example, the shape of the tooth 40 in FIG. 3 or 
the irregular shapes illustrated in FIG. 14. Clearly, such shapes are not rectangular, and 
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thus creating pseudo-pixels in accordance with the preferred embodiment will result in 
certain pseudo-pixels having bad pixels at the image edge. FIG. 4 illustrates bad pseudo- 
pixels 44a that contain pixels 42a, which are not part of the tooth image 40. Bad pixel 
routines used in accordance with a preferred embodiment to detect such pixels and 

5 disqualify them from further processing. For example, if 5% or more of the pixels within a 
pseudo-pixel are "bad" (e.g., containing reflections or other unwanted data), then such 
pseudo-pixels are disqualified. Though not shown, other pseudo-pixels might be 
disqualified if for example reflections from the light ports cause unwanted reflections in the 
other pseudo-pixels image 40. In a preferred embodiment, such pseudo-pixels are deleted 

1 0 from inclusion in the REFERENCE IMAGE. 

In a specific embodiment, the bad pixel routine need only be implemented when 
capturing and generating REAL IMAGES. In that process, conditions such as lighting and 
other variables can create unwanted artifacts that should be eliminated from processing. In 
addition, when cameras are used in the field,, one pixel might become defective over time; 

15 and REAL IMAGES later generated from the defective camera should be adjusted so that 
the pseudo-pixel which contains the bad pixel is^not counted or utilized. 

In another aspect, areas of the tooth for which the color is to be evaluated are 
predefined, allowing the analyzer program operating in accordance with this invention to 
batch-process the images. For example, a sample image can be loaded into an Area 

20 Selection Editor program module, where adjustments can be made to user-selected (or 

predefined) areas of the tooth image. These defined areas are then applied to each image in 
turn, and the pixel colors within each area are analyzed. In operation, following the image 
capture in one embodiment the method of this invention proceeds to automatically select the 
area(s) of the sample for analysis, for example, by applying a general rule to locate the 

25 edges of the tooth in the image, and apply a predefined segmentation of the remaining area 
for analysis. Preferably, the user is allowed to manually select an area of interest in the 
image, for example, using a computer mouse, as known in the art. 

In accordance with one embodiment, following the detection of the correct areas for 
analysis (i.e., excluding edges, light reflections and others), the selected area is divided by 

30 using, for example a grid overlay, as shown in Fig. 9. As known, each shade has a varying 
color content from top to bottom. Therefore, in accordance with this embodiment a more 
accurate analysis of the entire surface of interest can be made if color determination and 
matching is applied to the individual cells-bf the grid, as compared with corresponding area 
cells of the stored color reference model for'each^shade guide. 

35 Following this stage, in a preferred embodiment before analyzing the area, various . 

filtering operations can be applied to the image, as known in the art. For example, filtering 
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is applied to eliminate abnormalities such as lamp reflections or dark spots. In addition, 
maximum, minimum and average values for the R, G and B components can be determined 
over the area of interest arid used to, for example, limit the variation from the average value 
to halfway to the maximum and minimum value's.' This simple filtering operation has 

5 shown satisfactory results in actual testing; althaugh alternative or additional filtering 
operations can be applied, as known in the art in order to obtain a standard image. 

In the next step of the method, a SNAPSHOT of a patient's tooth is taken by the 
camera. Next, the digital image- of the SNAPSHOT is scaled, if necessary, to approximate 
the size of the corresponding REFERENCE IMAGE. In the preferred embodiment using 

10 pseudo-pixels SNAPSHOT pixels are next processed into pseudo-pixels resulting in a 
REAL IMAGE containing pseudo-pixels, which substantially correspond to REFERENCE 
IMAGE pseudo-pixels. A bad pixel routine preferably processes the REAL IMAGE to 
delete REAL IMAGE pseudo-pixels containing a bad pixel. As above, the bad pixel routine 
is particularly important at the edges of the tooth image within the SNAPSHOT, where 

1 5 some pixels will certainly contain background (unless the camera and optics are arranged to 
capture only the tooth; however this is not efficient since effective matching between the 
REAL IMAGE and the REFERENCE IMAGE occurs when a larger area of the tooth is 
used in the comparison algorithms, which are defined in further detail below. 

In a subsequent step, the REAL IMAGE is compared (i.e., correlated) to each 

20 REFERENCE IMAGE in the database (e.g., there could be sixteen REFERENCE IMAGES 
corresponding to the A1-A4, B1-B4, C1-C4 and D2-D4 Vita Shades) via the correlation 
algorithm (hereinafter "Correlation Algorithm") described below. In this step, each pseudo- 
pixel of the REAL IMAGE is compared to each pseudo-pixel of the REFERENCE IMAGE; 
and a composite match number ("CMN") ;'is created indicating how well the REAL IMAGE 

25 matches to that REFERENCE IMAGE. The composite match numbers are compared to 
one another and one of the REFERENCE IMAGES is selected as the "best fit" match to the 
REAL IMAGE. 

There is potentially a problem associated with the bad pixel routine and subsequent 
correlation between REAL IMAGES and the series of REFERENCE IMAGES. As 

30 described above, in a specific embodiment, when there is a bad pixel in any pseudo-pixel, 
all other pseudo-pixels of the same spatial location are discarded. This can become a 
problem in a case when every (or even most) pseudo-pixel is disqualified, resulting in a 
situation where no meaningful comparison can be made. Accordingly, in a preferred 
embodiment, images are correlated on the basis of mathematical measure, Le., an average, 

35 that is functionally dependent upon how many pseudo-pixels remain in an image (REAL or 
REFERENCE). That is, for any. given correlation between a REAL IMAGE and a 
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REFERENCE IMAGE, the number of pseudo-pixels for that comparison are used as a ratio 
for comparison to other correlation. This aspect of the invention is described in more detail 
below. 

In an alternative embodiment, thd averaging technique discussed above is used only 

5 when, for example, more than 20-25% the pseudo-pixel's are disqualified for all 

comparisons. Accordingly, so long as there is a sufficient number of remaining pseudo- 
pixels for comparison, a direct comparison of these pixels can be made without resorting to 
averages. In a specific embodiment, a sufficient number is deemed to be about 75-80% of 
the total number of pseudo-pixels available for comparison. Other ratios can be used in 

10 alternate embodiments. 

Bad pixel routines are generally known in the art and thus need not be described in 
much detail. It is sufficient to note that in accordance with this invention a pixel is 
determined to be "bad' 1 if its light intensity or color values deviate by more than a certain 
predefined percentage from adjacent pixels known to be "good". For example, if a pixel 

15 deviates by more than 30% from the light intensity of the neighboring 8 pixels, there is a 
good likelihood that this deviation is anomalous, i.e., due to a bad camera element or 
corresponding to an image border, and has to be discarded. 

In a preferred embodiment, a pseudo-pixel is validated only when it contains less 
than a certain percentage, i.e., about 5%, bad pixels of the total pixels making up the 

20 pseudo-pixel. Preferably, bad pixels are also not used in the statistical characterization 
(e.g., RGB) of the pseudo-pixel. Accordingly, in this embodiment if more than about 5% 
bad pixels exist for a pseudo-pixel, the pseudo-pixel is not used in further processing. 

Correlation Algorithms ' - : . f v 

25 ^ In a preferred embodiment, the Correlation Algorithm of the present invention 

operates as follows. Each REFERENCE IMAGE is actually a matrix of vectors, each 
vector corresponding to a pseudo-pixel. By way of example, the REFERENCE IMAGE 
corresponding to the Al Vita Shade can be assigned as vector Z AJ . For the sixteen Vita 
Shade guide, the remaining fifteen shades for example each have a REFERENCE IMAGE 

30 too, e.g., Z^, Z^, etc. 

Each REFERENCE IMAGE vector "Z" - corresponding to shade guide or porcelain 
"X" - thus has data similar to the following matrix: 

35 
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where each of the pseudo-pixels"PP" has three values for each of R, G and B values of the 
pseudo-pixel (actually, the RGB values are the statistically computed (e.g., averaged) 
composition of the images in the series for that REFERENCE IMAGE, if available). The 
subscript "x" refers to the appropriate shade, e.g., "Al". Subscripts 1-n define separate 
pseudo-pixels in the REFERENCE IMAGE. Those skilled in the art will appreciate that 
additional, other or different data can make up each vector, including hue data for each 
pseudo-pixel. Additionally, other vectors can be considered and processed in the 
correlation, such as hue and RGB values. 

In a typical example, each REFERENCE IMAGE might have 20x20 pseudo-pixels 
which define the REFERENCE IMAGE. Therefore, "n" in the above matrix is 400. 

When a REAL IMAGE "I" is generated, in accordance with this invention it too is 
arranged as a matrix of similar form, with each pseudo-pixel "PI" of the REAL IMAGE 
being a vector of RGB form (or, like above, containing other or additional factors such as 
hue): 













3, 




PI x,2 




R i,2 


G,,2 






PI x,l 




R K3 


Gf.3. 






P *x, n 











In a specific embodiment., the Correlation Algorithm used in accordance with the 
present invention computes a measure of closeness, i.e., a composite match number 
("CMN") through the following relationship: 
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5 „ f 

cmn x = z vK, - x,*f + K, - + K - 3 J 

Once the CMN number is computed for each tooth shade, as shown in the example 
1Q above, a search is then conducted for the lowest CMN X to find the best fit REFERENCE 
IMAGE for the REAL IMAGE. That is, the tooth shade or porcelain "X" is identified for 
the lowest associated value of CMN. As noted above, if there is more than a certain 
percentage of bad pixels in any pseudo-pixel q for either the REFERENCE IMAGE or the 
REAL IMAGE, in a preferred embodiment that pseudo-pixel is not used in the valuation of 
^ CMN. For example, in accordance with the present invention it is acceptable to determine 
CMN without the q-th pseudo-pixel; however, every other concurrent q-th pseudo-pixel 
valuation of CMN X in identifying the comppsite^rnatch number is also discarded, so that 
CMNs for all tooth shades can be compared correctly. 

As noted, the Correlation Algorithm of the embodiment illustrated above preferably 
2q uses a bad pixel routine to disqualify bad pseudo-pixels. It was noted already that this can 
create problems in certain situations. Accordingly, in a preferred embodiment, the 
following alternative algorithm can instead be used: 

25 x £i yv P count x J \ Pcount x J \ P count x ) 

In this embodiment for the computation of CMN Pcount x corresponds to the number 
, of common pseudo-pixels found between the REAL IMAGE and the vector Z x . Note that 
Pcount x can be different for each CMN correlation. For example, if the REAL IMAGE has 

30 400 pseudo-pixels, all good, and REFERENCE IMAGE for Al has 399 pseudo-pixels (e.g., 
one bad pseudo-pixel identified in the bad pixel routine), then Pcount A1 is 399. If however 
the REFERENCE IMAGE for B4 has 256 pseudo-pixels, then Pcount B4 is 256. If in the 
same example the REAL IMAGE has 256 valid pseudo-pixels - and in the unlikely event 
that the disqualified REAL IMAGE pseudo-pixels overlap with the coordinates of 

35 disqualified pseudo-pixels in the REFERENCE rIMAGE - then Pcount B4 is still 256; 

however Pcount AI is also 256 (assuming that the 'one bad pixel of REFERENCE IMAGE Al 
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corresponds to one of the disqualified pseudo-pixels in the REAL IMAGE. If the one bad 
pseudo-pixel in REFERENCE IMAGE Al does not correspond to coordinates of one of the 
disqualified pseudo-pixels of the REAL IMAGE, a more likely event, then Pcount Al is also 
■ 255. " 

5 Those skilled in the art will appreeiate^that isolating the measure of closeness CMN 

in one of the above equations can also be determined without the square root operation - as 
a minimum composite match number will still be identified for the same functional 
conditions and/or data. 

In accordance with the specific embodiment described above, the process of 
1 0 determining Pcourit x can be made at any point. In a preferred embodiment, this process is 
initiated only after a certain percentage of pseudo-pixels are disqualified. For example, if 
after the completion of the bad pixel routine there remain 300 pseudo-pixels for comparison 
(in the example that 400 pseudo-pixels exist in each of the REFERENCE and REAL 
IMAGES), then a straight comparison can be made without the use of the Pcount x 
15 adjustment, because a significant percentage of the images can be compared (defining 75% 
as "significant"; other percentages can be used). Note that it is likely that many of the bad 
pixel areas in images will overlap, such as at the edge of the image, where tooth shape 
variations occur often, and at locations such as reflection areas (e.g., areas which specularly 
reflect light energy to the camera), which are likely to be similar given that the illumination 
20 source is generally fixed for each image acquisition. 

Those skilled in the art will appreciate that variations to the above-described 
methodology may occur without departing frorft the scope of the invention. For example, 
other color detection techniques can be used to characterize tooth color within a pseudo- 
pixel. In one example, colorimetric 'laser-diode 1 ', measurements can be used to generate a 
25 reflection trace for each pseudo-pixel. In such measurements, a laser diode is "scanned" in 
wavelength so that a laser beam, scanned through a range of wavelengths, reflects off of 
each pseudo-pixel. This spectrophotometric-like trace information (for example containing 
reflectance per wavelength) can be collated with other such traces for other pseudo-pixels to 
generate a vector of such information. As above, correlation between real and reference 
30 vectors is used to determine a best-fit color match. 

In accordance with another embodiment of the present invention, the camera used 
. for capturing images has a focal plane array with fewer detector elements as compared to 
typical high resolution arrays (for example those arrays with 640x480 elements, or 
megapixel digital cameras). For example, in one embodiment of the invention an array has 
35 a relatively small number of detectors, i.e., 20x20, 60x40 or others. Such a camera can 
alleviate the need for pseudo-pixels, as defined above, since each real pixel generated from 
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each detector covers a relatively large area of the tooth image. In effect, such a camera 
generates "pseudo-pixels" in each digital frame image. Since fewer detector elements are 
used in this embodiment, it will be appreciated that the camera's overall cost can be" 
reduced. Those skilled in the art will appreciate that a similar effect may be obtained in an 

5 alternative embodiment by using magnification optics that only utilizes a small portion of 
the camera's array in obtaining the image; however such an approach wastes pixel 
information which has already been collected. 

Those skilled in the art will appreciate that other closeness measures can be used 
instead of the algorithms described above to achieve a similar result. By way of example, 

10 other techniques of measuring similarity between two data sets can be used in determining a 
measure of how similar or cloase two data sets are. For example, one can arrange a data set 
as a vector so that the correlation coefficient is determined as the cosine of the angle 
between data vectors. Cross-correlation functions or matched filtering can also be used 
beneficially. The interested reader is directed to any number of books on digital image and 

15 signal processing, such as, for example, Netravali and Haskell, "Digital Pictures, 

Representation and Compression," Plenum Press, 1988. Sections 1.1; 1.2; 1.3; 1.8; 1.9; 2.2; 
3,2; and 3.3 of this book are incorporated herewith by reference for background purposes. 

In still another embodiment, the shape of the grid of pseudo-pixels defining each 
tooth is selected in a manner dependent upon how the tooth is positioned in the mouth. For 

20 example, an incisor tooth very nearly maps to a shade guide; however, with reference to 
Fig. 14, a posterior tooth does not, particularly relative to the image capture position of the 
camera. Further, it will be appreciated that for purposes of improving the appearance of a 
smile, anterior teeth are considerably more important than those in the posterior of the 
mouth. Thus, in a preferred embodiment it is desirable to provide more close matches, and 

25 correspondingly more dense and accurate grid patterns for the anterior teeth. Accordingly, 
in a specific embodiment, grid shapes and correlation algorithms can depend upon tooth 
orientation within the mouth and/or upon generalized tooth shape. In particular, anterior 
teeth will have a (proportionately) larger number of pseudo-pixels than teeth in the back of 
the mouth for the same surface area. 

30 An image of a tooth or tooth shade may be analyzed by a flood fill algorithm to find 

.the edges of the target tooth or tooth shade. Using this algorithm, from a point in the image 
(e.g., in the SNAPSHOT) known to be in the tooth or tooth shade, adjacent pixels are 
considered only if that pixel is in a valid color range. The maximum extent is then recorded 
for the search in the X and Y directions, forming the outer limits of the grid. In addition; 

35 contrast changes in groups of pixels can be considered and used to determine the extent; but 
a black border is easy to find. 
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In accordance with another important embodiment, "tooth shades", as used above, 
are not used per se in defining a patient's tooth color. Rather, in a specific embodiment, a 
grid of pseudo-pixels is generated for the patient's tooth; and these (pseudo-)pixels define 
the porcelain for respective regions of the tooth. Pseudo-pixels are not actually required; 
and actual pixels can also be used in this manner to define porcelain characteristics for each 
spatial location in the mouth. Reconstructive tooth material is then specified per pixel or 
pseudo-pixel. A data file generated from the SNAPSHOT or REAL IMAGE is then 
processed to specify reconstructive materials with spatial precision, as defined by pixels or 
pseudo-pixels. An acceptable quantification of RGB values, for example, can be used to 
tolerance both the measurement and materials specification. For example, by associating 
error bars with each measure - e.g., R +/- AR, G +/- AG, B +/- AB, where the A quantities 
are defined within certain practical tolerance limits - reasonable tolerances can be achieved. 
Tooth shades operate similarly in that each shade results in a quantized color difference 
from every other shade; and thus the aforementioned tolerancing technique provides similar 
accuracy to the above-described correlation algorithm. Unlike the tooth shade approach, 
however, spatial accuracy for any given reconstructive tooth is generally determined only 
by the number of pixels or pseudo-pixels used in the reconstructive tooth manufacture; 
whereas a tooth shade has predefined color gradients defining the tooth shade regardless of 
the numbers of pixels or pseudo-pixels used. It will be appreciated that this approach 
avoids the use of tooth shades at all, along with the possible confusion created by the 
existence of alternate tooth shade guides. 

The Color Measuring System and its Components 

Cameras of the type required for use with this invention are generally known in the 
art and include, for example: INSIGHT™, manufactured in San Carlos, California; 
CYGNASCOPE™ offered by Cygnus Instruments, Inc., Goleta, California; 
VISTACAM™ and others. In a preferred embodiment, the system of this invention uses a 
Welch- Allyn brand camera. Generally, it is desired to use camera systems offering full- 
color imagery, which are capable of capturing a range of sizes, i.e., from the size of a 
typical patient's tooth preferably to images of the patient's whole smile. 

In the preferred embodiment, it is advantageous for the camera to supply a minimum 
640 x 480 pixel image to the PC software 24 bits per pixel (i.e., 8 bits for each of the red, 
green and blue (RGB) components), or preferably 30 bits per pixel. In a specific example, 
the system of this invention uses ALARIS QUICK VIDEO TRANSPORT frame grabber, 
providing digitized images with at least 24 bits, resolution. More specifically, the software 
can use a Twain protocol interface, as Iqipwn in the art, which allows other cameras and 
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frame grabbers to be tested without the need for a change of software. Preferably, images 
captured by the camera are displayed on a monitor screen to provide instantaneous feedback 
to the system operator. 

In another preferred embodiment, the resolution of the camera is specified in terms 

5 of the Minimum Resolvable Color Difference that the complete system is able to achieve. 
This can be specified, for example, as the two RGB values of the two closest shades the 
system is required to differentiate. For example, the Chromoscop Shades 430 and 440 can 
be used to this end. In general, it is envisioned that the system should be able to 
differentiate between about 80 or more different shades. Another requirement is that the 

10 system should be able to produce repeatable images. That is to say that images of the same 
tooth taken at the same session should not have a Ai of not more than 0.1, which is the 
amount needed for the eye to perceive a difference. In a specific embodiment, the camera 
. used in the system of this invention is a CMOS imager. 

In terms of its physical setup, a stand-alone camera containing its own light source 

1 5 can be used, as explained below. A number of alternate embodiments are available in this 
regard. For example, in one embodiment the camera can be battery powered. In this 
embodiment, the camera can sits on a holder containing an inductive battery charger when it 
is not in use. In another embodiment, when mounted on the charger the camera can be 
coupled via an isolation sleeve (to be explained below) to a calibration target, for example, 

20 made of porcelain. 

In a specific embodiment the output of the camera is supplied to a digitizer (such as 
a Sony digitizer) enabling convenient digital storage of the image. As noted above, the 
output of the camera can also be supplied to a frame grabber in a PC. Both options can be 
used in a specific embodiment. In another embodiment, the output of the camera can be 

25 supplied directly to a monitor (preferably positioned close to a surgery chair) and provide a 
digital output to a PC, which then need not be close to the patient. As known in the art, the 
output could be USB-type, or IEEE 1394. 

In accordance with a preferred embodiment the digital output of the camera also 
provides the opportunity to control the camera from a PC. Finally, in a preferred 

30 embodiment it is desirable to control the camera so as to use it in two modes, i.e., normal 
image - for the mouth and full face shots; and analysis image - in which color balance and 
automatic functions are disabled for tooth and calibration image, as described below. 

Turning now to the drawings, FIG. 1 shows a system 10 constructed according to a 
preferred embodiment of the invention. A solid state (or intra-oral) camera 12 connects to a 

35 computer 14 via a PC card 16 to capture images through a >vand 1 8. The solid state camera 
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12 includes a detector array 12a including an array of detector elements 12b, which generate 
pixels in the digital images (e.g., SNAPSHOTS) captured by the camera 12. 

Through one of several known mechanisms, internal optics within the wand 18 
and/or camera 12 permit the capture of an image of a target object 20 (for purposes of 

5 illustration, target object 20 is shown grossly over-sized as compared to other elements in 
FIG. 1) by the array 12a. By way of example, relay optics 18a within the wand relays an 
image to the array 12a. A protection sleeve 22, discussed in further detail below (also 
grossly oversized for purposes of illustration), preferably extends from the wand 18. As 
shown, the optics provide an optical conjugate between the array 12a and the target object 

10 20 through well-known imaging techniques. Light captured from the target object 20 enters 
the wand 18 for transfer to the camera 12 through an entrance aperture window 26. 

The wand 18 generates light 19 to illuminate the target object 20 through light ports 
28. Preferably, light from the outside 30 of a sleeve 22 is not permitted to illuminate the 
object 20 so that control is maintained; and thus the sleeve 22 shields the target area 20 

15 from illumination by outside sources 30 (e.g.,, ambient room lighting). 

An aperture 32 within the center of the end piece 34 of the sleeve 22 is where the 
tooth or tooth shade are placed so that a SNAPSHOT (i.e., a digital image of the tooth or 
tooth shade) can be made. As discussed above, these SNAPSHOTS are processed to form 
REAL IMAGES (from real teeth) or REFERENCE IMAGES (from tooth shades or 

20 porcelains, etc.). 

A black border 36 around the aperture 23 provides a good reference around which 
the tooth or tooth shade are discernible within the digital image of the target area 20. The 
remaining area 38 about the border 36 and within the end piece 34 is preferably a white 
reference sample, equally reflecting all light 19 from the light ports 28. 

25 Finally, again with reference to FIG. 1 , digital images from the camera 12 are sent to 

the computer 14; and processing software 50 within the computer 14 processes these images 
to generate, i.e., a CMN for each REAL IMAGE relative to the REFERENCE IMAGES. 
As discussed above, the software 50 processes the CMNs to locate the lowest value CMN, 
indicating a match; and communicates the associated shade of that lowest CMN to the user 

30 via signal line 52. As mentioned, other processing algorithms can be developed to 
determine a best-fit match without departing from the scope of the invention. 

A representative digital image 30 captured by the camera 12 is illustrated in FIG. 2, 
showing an image 36 s of the border 36, an image 38* of the reference sample 38, and a 
tooth image 40. The entire image 30 covers the target area 20 of FIG. 1. FIG. 2 also 

35 illustrates obvious regions 41 of the image 30 that would generate bad pixels since such 
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regions do not contain tooth imagery but rather other background imagery (e.g., the 
patient's gum). 

FIG. 3 shows a blow up image of the pixels 42 (only representative pixels 42 are 
shown), which make up the tooth image 40 of FIG. 2. In accord with the invention, these 

5 pixels are transformed into pseudo-pixels 44 of FlG. 4. Each pseudo-pixel 44 is made up of 
all real pixels within the area of the associated pseudo-pixel 44. Two pseudo-pixels 44b 
illustrate, for example, how a pseudo-pixel can be generated from nine real pixels 42. FIG. 
4 also illustrates that an image can be made from either a real tooth 40 (resulting in a REAL 
IMAGE) or a reference shade 40 (resulting in a REFERENCE IMAGE). REAL IMAGES 

1 0 and REFERENCE IMAGES are correlated to find the composite match number (CMN) as 
described above. 

FIG. 5 shows another embodiment of an end piece 98 used in accordance with a 
specific embodiment, that mounts to, or is made integrally with, the end of the sleeve (e.g., 
the sleeve 22, FIG. 1) and which has a series of tooth shades 100 disposed in the end plate 

15 98, so that for each target 102 (e.g., the tooth or tooth shade), all relevant manufacturing 
shades are provided in the same digital image, thereby preventing color contamination or 
other anomalies caused by time delay. As discussed above, when using the end piece 98 of 
this embodiment each shade 100 is processed as a REFERENCE IMAGE and the tooth 102 
is processed as a REAL IMAGE relative to those REFERENCE IMAGES to find a CMN. 

20 Preferably, a black border 104 surrounds the tooth aperture 106 and tooth 102. Also 
preferably, the remaining area 106 about the border 104 and in the end piece 98 is a 
reference area. By way of example, the reference area 106 is a white reflecting region with 
can be sampled by detectors that image that region 106. Further examples of the use of 
reference area are discussed below. " ^ 

25 

Isolation Sleeve 

In a preferred embodiment, an isolation sleeve is used to reduce variations in the 
images captured and processed by the system, and in particular to eliminate light 
contamination from external sources. In addition, the isolation sleeve preferably keeps the 

30 reference shade and the actual tooth at a set distance from the illumination source and the 
camera optics. The sleeve also preferably sets the angle of illumination between the source 
and the tooth so as to reduce reflections. More particularly, the REFERENCE IMAGES 
and the REAL IMAGE are preferably taken at the same illumination intensities, at 
approximately the same distance, and without substantial specular reflections from the * 

35 source. To this end, in a preferred embodiment the sleeve shields the camera detector from 
imaging outside light and instead utilizes internally generated light (i.e., internal to the 
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camera, for example, or associated with an intra-bral wand attached to the camera) that can 
be controlled. The sides (or side) of the sleeve are coated in a specific embodiment with a 
black material (e.g., a paint or a black mat paper, or black felt), which reduces reflections 
along the sleeve to the tooth or reference shade. As used herein, "target area" refers to the 

5 image gathering location that the system of the invention (i.e., that region captured by the 
camera's detectors), including the REAL or REFERENCE IMAGE, as defined above. 

FIG. 6 shows one end of wand 130 and a sleeve 1 32 constructed according to a 
specific embodiment of the invention. As in the example shown in FIG. 1 above, the wand 
103 connects to a solid state camera (not shown, for purposes of illustration) to collect 

1 0 digital images of the target region 134 at the end of the sleeve 132. By way of example, the 
target region 134 includes an aperture (not shown) for imaging a tooth therein. Light 135 
from the camera or wand 130 exits the wand 1 30 at optical aperture 136 to illuminate the 
target region 134. 

In a specific embodiment illustrated in FIG. 6, the sleeve 132 used in the camera 
15 system of the present invention includes an accordion-like exterior, which permits soft 
placement of the end of the sleeve onto the patient's tooth. Preferably, such a sleeve is not 
entirely rigid so that the sleeve 132 can make contact without concerns about damaging the 
tooth. The outer portion of the sleeve 132 : in this embodiment thus acts similar to a spring, 
and an inner structural member within the sleeve sets the final source-to-tooth distance once 

20 the outer sleeve/spring compresses to the desired location. 

As shown in FIG. 6, the accordion-like sleeve 132 compresses between the target 
region 134 and the wand 130, as shown by compression arrow 138. In operation, a user of 
the wand/sleeve 130/132 pushes the sleeve 132 to the patient's tooth, and the sleeve 132 
compresses to provide comfortable (i.e., non-rigid contact). In a preferred embodiment, the 

25 sleeve 132 is spring-like to provide some force opposing to compression. This force 
increases until there is an interaction between the sleeve 132, and/or the end piece of the 
sleeve 132 (i.e., the part of the sleeve at the target region 134), and the rigid structural 
member 140 within the sleeve 132. The member 140 stops compression at a fixed location 
so that a set distance is achieved from the aperture 136 and the target region 140; arid so that 

30 a repeatable image size is attained. 

Illumination 

As noted, in a preferred embodiment, the camera system of this invention includes a 
light source that illuminates the target area. In a specific embodiment the sleeve 132 can be 
35 made to rotate so that images are gathered from difficult locations in the mouth. Preferably, 
the light source is tied to fiber optics which rotate with the sleeve so that regardless of 
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sleeve position the source-to-target area remains approximately fixed. In another aspect, 
the camera includes optics, such as image collection optics and/or an entrance window. In 
system embodiment including this feature, the camera optics is tied to fiber optics, so that 
images are captured effectively regardless of the position of the sleeve. 

5 In another aspect, the sleeve used with the dental camera system of the present 

invention incorporates imaging optics which relay the tooth image through a Lyot stop, to 
prevent passage of unwanted light energy to the camera detectors. In a specific 
embodiment, the sleeve incorporates baffling - such as "two-bounce" optical stray light 
baffling - to reduce or substantially eliminate stray light from external sources to the desired 

10 tooth image area. 

FIG. 7 shows illumination arrangement wherein the source 150 of the illuminating 
wand 152 (connected to the solid state camera; not shown) is angled . from the target area 
154. As shown, the sleeve 156 connected to the wand 152 is arranged adjacent to a 
patient's tooth 158, so that digital images can be taken of r the tooth 158 (according to the 

15 practices discussed herein) through the wand's optical entrance aperture 160. 

For purposes of illustration, FIG. 7 also shows how light 162 emitting from the 
source 150 travels in a generally specular direction 164, reflecting off the tooth 158 into a 
direction 166 that is away from the aperture 160. Those skilled in the art will appreciate 
that scattering does occur off the tooth 158 and into the aperture 160, but that the 

20 arrangement eliminates some unwanted reflections into the aperture. Light captured 
through the aperture 160 is relayed by optics (e.g., fibers and/or relay lenses) to the 
camera's focal plane (not shown) to generate digital images of the tooth 158. 

FIG. 8 shows another embodiment of a sleeve 1 68 constructed according to the 
invention to reduce passage of light 171 into the wand's entrance aperture 169 from sources 

25 170 away from the target object (e.g., the tooth 172). The sleeve 168 is especially useful in 
imaging the tooth 172 without adjacent proximity to the sleeve 168, as illustrated in FIG. 8. 
The sleeve 168 includes baffles 168a known in the art, which require at least one "bounce" 
and preferably two bounces of the light 171 prior to gaining access to the entrance aperture 
169, thereby significantly attenuating "out of field" sources 170 (i.e., those unwanted 

30 sources which might influence the color measure, e.g., room lighting). For simplicity, in 
this illustration the wand and solid state camera are not shown. Baffles 168a can be placed 
within other sleeves shown herein to improve qut-of-field stray light rejection. 

As noted above, in order to ensure high-quality images which do not depend on the 
specific lightning conditions at the time the pictures are taken, in accordance with this 

35 invention it is important to minimize the influence of such factors. Accordingly, in another, 
specific embodiment, the sleeve used in the dental camera system of this invention includes 
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a reference sample disposed at the end of the sleeve, near the target area, so that: (a) color 
comparison information can be obtained; and/or (b) the camera has sufficient reflective 
surfaces from which to effectively trigger the camera's auto-brightness and/or auto-color 
features. Specifically, as to feature (b), certain cameras available on the market include 

5 electronics and software which automatically adjust brightness and/or color in a digital 
image. Preferably, in this embodiment such features are disabled. In the event they are 
operating, however, the reference sample is sized so that a sufficient reflection area is 
generated at the end of the sleeve, whereby the camera can operate to capture good color 
images. By way of example, as to feature (b) above, the sample should be sized so that 

10 RGB values vary in a controlled or calibrated manner throughout the reasonable tooth shade 
reflections (e.g., throughout all VITA Shades). 

As to feature (a) above, one preferred embodiment utilizes the reference sample to 
obtain a color reference used to reduce variations in the digital image. "Color*' is based on 
"reflection" - that is, what the camera sees aUhe target area is based on reflection of 

15 whatever source illuminates the target. With the sleeve used in a preferred embodiment, the 
source is limited to the camera's illuminating source, thereby eliminating other sources and 
color variations that are not controllable (e.g., the ambient lighting in a building). It is well 
known that a black body absorbs visible light; and a white reflects the light. In one 
embodiment, therefore, the reference sample is as white as possible so that it exhibits very 

20 little color (and particularly, the sample reflects light equally in the visible light range from 
between about 400nm to 800nm). When using a white reference sample in accordance with 
this embodiment of the invention, the following process occurs: 

1) Compute average RGB values (and/or other image variables such as hue) for 
several or all pixels imaged of the reference sample. This computed average is 

25 referred to as REF RGB. 

2) For each digital image (i.e., for both REFERENCE IMAGES and REAL 
IMAGES), subtract REF RGB from RGB pixel values for that image. Keep track of 
the sign (i.e., positive or negative) of the result. The result is referred to as "REF 
RELATIVE RGB" (i.e., image RGB - REF RGB). 

30 3) Utilize a correlation algorithm, such as described above, (i.e., for the CMN) 

on REF RELATIVE RGBs as opposed to absolute RGB values (for the pseudo- 
pixels). Note that when the RSS (i.e., subtract and square) of the CMN algorithm is 
computed, the sign of the REF RELATIVE RGB matters. For example, if the 
REFERENCE IMAGE RGB is -.l,-.5,-.6, and the REAL IMAGE RGB is .1, .7, .9, 

35 then this REAL IMAGE has quite a bit more color difference (compared to the REF 
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IMAGE, which is the important quantity) than, e.g., a REAL IMAGE with -.05, .2, 
.1. Afterwards, the square of the RSS destroys the : sign importance. 
The reference sample compensation algorithm described above is used in a specific 
embodiment to compensate for certain variations. Thus, the auto-brightness feature of 

5 certain cameras changes the color of the light emitted from the camera (this is sometimes 
referred to in the art as the color temperature). Unfortunately, the emitted RGB is not 
known except for the reference sample, which reflects the source light back to the camera 
detectors. A good reference sample will thus reflect nearly all colors equally. Since on is 
interested in differences between REAL IMAGES and REFERENCE IMAGES, the real 

10 concern involves differences and not absolute colors. 

The reference sample compensation thus also compensates for image acquisition 
changes which occur over time. For example, the source may emit more or less light, over 
time, even over several minutes or hours; and it would be desirable to eliminate such 
variations to increase the sensitivity to color comparisons. The passage of time during an 

1 5 image acquisition sequence only adds to the variability in the measurement: the source color 
temperature may change, the detector sensitivity or gain may change, etc. With the above 
reference sample adjustment, if for example one has a perfect white tooth, then its REF 
RELATIVE RGB is 0,0,0. Assuming that a tooth shade exists that was also perfectly white, 
it would have a REF RELATIVE RGB of 0,0,0 - creating a match. 

20 In another embodiment, the white reference is integrated to find REF RGB, per 

frame. REF RGB is then subtracted from each pixel RGB in the image (or the other way, 
i.e., image RGB subtracted from REF RGB, as long as consistent throughout every 
measurement). In another embodiment, REF RGB is subtracted from pseudo-pixels; but 
preferably REF RGB is subtracted from real pixel RGBs. 

25 In another embodiment, the sleeve used with the dental camera system of the present 

invention is formed in the following way. At the end of the sleeve, a central aperture exists 
preferably in the middle of the target area (e.g., an object such as a tooth or shade is placed 
at the aperture). Surrounding the central aperture is a black border, to provide high contrast 
at the edges of the target object. A target object such as a tooth is thus readily defined and 

30 discerned within the black border. The aperture also fixes the size of the image for the tooth 
or tooth shade. Some or all of the remaining area at the end of the sleeve (captured by the 
camera detectors) in the target area is a white reference sample. 

. The amount of white reference sample in the target area can be chosen 
experimentally. By way of example, the average mid point in the auto-brightness (if 

35 operating) is obtained so that on average REF RGB changes little. The size of the white 
sample in the target area is adjusted in area until REF RGB is minimized for all shade 
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values, e.g., A1-A4, B1-B4 and so on. With more black in the image due to less reference 
sample area, the auto brightness control on the camera (if applicable) adjusts to higher gain 
to 'balance 1 the intensity of the image, causing the reference and sample parts to saturate in 
the red and green. 

5 As noted above, in a specific embodiment, the walls of the sleeve are selected mat 

black to eliminate reflections, but the facing plate containing the target area is bright to 
force the camera gain downwards, out of non-linear color operability. In. this embodiment 
preferably the camera's auto features are turned off (and at least any white balance is set to 
manual). In one specific embodiment, the reference sample is made of a light colored felt 
10 material. In alternative embodiments, the sleeve walls are made of a felt material. In these 
embodiments, the felt material has elements which extend away from the material 
producing more of a lambertian surface. Such surfaces are preferred as they reduce 
unwanted specular reflections. Also, the sample reference produces an excellent image 
when it is not specular. Alternatively ; a black veloiir paper can be used in the sleeve, such 
15 as by JL Hammett Co. ' • ' ' ; ' > 

FIG. 10 illustrates a non-contact re-imaging system 250 used in accordance with 
another embodiment of the present invention to image a target tooth 252 without contact 
between the tooth 252 and a sleeve 254. Optics 255 reimage the tooth 252 internal to the 
sleeve 254, at internal image 256, and a Lyot stop 258 is used to reduce unwanted stray 
20 light entering the aperture 260 to the camera (not shown). 

FIG. 1 1 illustrates a non-contact re-imaging system 300 used to image a target tooth 
302 to a digital camera 304, and without contact between the tooth 302 and the system 300 
or camera 304. Although this reimaging system 300 can be made in several forms, in one 
embodiment the system 300 includes a sleeve 306 that permits hand-held manipulation into 
25 a patient's mouth to capture the digital image of the tooth 302. By way of example, optics 
308 reimage the tooth 302 internal to the sleeve 306, at internal image 310, and a stop 312 
is used for color referencing in analyzing the tooth color. Stop 312 forms an aperture 
defined by edge 312a. FIG. 12 illustrates one digital image 320, as taken by camera 304, of 
the tooth 302 and the inside view of stop 312. The region 322 defines that region inside the 
30 patient's mouth that is not the patient's tooth 3 1 0. Region 324 consists of a color reference 
which is used as described herein to relate and compare to color pixels of the digital image 
of the tooth image 3 10, so as to better define tooth color. 

Those skilled in the art will appreciate that $ reimaging system such as in FIG. 1 1 
can be made in several ways. By way of exumple; system 350 of FIG. 13 shows one system 
35 of the invention to reimage a tooth 352 to an internal image 354 for reimaging into a digital 
camera 356. As above, camera 3.56 takes SNAPSHOTS of the tooth 352, for color analysis. 
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Optical element 358 images the tooth into optical fiber bundle 360, which relays the image 
from one side 360a to the other side 360b of the bundle 360, as known in the art. Optical 
elements 362 provide for reimaging to form the internal image 354 at the stop 364. As 
above, the stop 364 has a reference color disposed thereon, facing camera 356, so that a 

5 reference color image is attained such as in FIG. 12. Fiber optic bundle 366 relays the 

image 354 from side 366a to 366b, and exit optics 368 provides for relaying the tooth image 
to the camera 356. One convenient feature of system 350 is that fibers 366, 360 can be 
flexible; and a sleeve 370 can support these elements to provide a hand-held wand that can 
be inserted into a patient's mouth to acquire the image. Camera 356 can provide its own 

10 light source 356a which generates light 356b back along the optical path taken by tooth 
image 354. The advantage of this is that source 356a can be carefully selected for its color 
characteristics to facilitate tooth color detection; and further light 356b can illuminate stop 
364 inside the sleeve or wand 370 so that the camera 356 can detect and compare its color 
to the tooth's color image 354. . 

15 Tooth Restorative Processing 

FIG. 14 shows certain tooth restorations and decays, which illustrations can help 
understand more fully aspects of the invention discussed above. A healthy tooth, free from 
any decay with no current restoration ("restoration" is any part of a tooth that is replaced 
with a material that allows the tooth to remain in the mouth as a functioning and whole 

20 structure) is referred to as a "virgin" tooth. Despite advances in preventative treatment and 
services (fluoridated water, fluoride treatments, sealants - which are unfilled resins that are 
bonded into deep grooves of posterior or back teeth to prevent decay in those areas),. 50% of 
American children by age 12 have occlusal decay ( decay in the top, or biting surface) in 
permanent molars which erupted or came into their mouths at age 6. 

25 Typical progression of decay in a tooth is as follows: Following C.V. Black's 

classifications, a tooth can require a restoration in the following positions: 

CLASS 1 - occlusal area, that is, within only the top or biting surface of the tooth, 
usually beginning in the.groove or crevice. This term is used only for posterior (back) teeth, 
i.e., molars and premolars. 

30 CLASS 2 - area made up of occlusal and one or more sides of the tooth, either 

mesial (wall towards front) and/pr distal (wall towards back) and or buccal(wall towards 
cheek) and or lingual(wall towards tongue) so that a class 2 restoration may be termed 
"MO" (mesial-occlusal), "MOD", "OB", etc. ; 

CLASS 3 - area of an anterior, or front, tooth involving only an interproximal wall, 

35 that is mesial or distal, areas that face neighboring teeth. 
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CLASS 4 - area of an anterior tooth involving the incisal (bottom) edge and an 
interpoximal wall. 

CLASS 5 - area of any tooth on only the buccal or lingual wall 
CLASS 6 - area of a tooth involving the cusp tip ( cusp being the highest point of 
5 the tooth , like the mountain peak; this would apply to canines, premolars, and molars) 

Once decay is detected, through clinical examination, radiographs, etc., the decayed 
portion of the tooth needs to be removed; ' This is achieved through the use of a handpiece 
(drill). Once excavation of decay is complete, the remaining tooth structure is evaluated for 
restoration possibilities. A "filling" is placed if 50% or more of the tooth remains, with the 
10 stress-bearing areas of the tooth remaining intact (such as cusps and walls of the tooth 
which are active in biting and chewing process). If these areas of the tooth are 
compromised, a laboratory-processed restoration is required. 

Consider a specific example, in which it is assumed that the tooth needs a 
restoration, which will be placed right in the office, say a MO on a molar. The choice of 
15 materials is made, which could be either amalgam (silver, not done much anymore),or 
composite or ceromer, which are tooth-colored direct materials (Matrix of ground plastic 
and /or glass in a bis- GMA resin). A shade needs to be selected for the material, such as 
described herein in accord with the invention. The tooth is cleaned and isolated from saliva 
and blood by use of cotton rolls, matrix bands, possibly a rubber dam. The tooth surface is 
20 etched with a cleanser (typically 37% hydrophosphuric acid), rinsed, and treated with an 
adhesive, which is bonded to the tooth by use of a curing light - a light with a wand 
attachment that is about 1 1 - 13 cm in width and emits a light in the range of 400-500 
nanometers. The material is then placed into the cavity by hand instruments or via 
dispensing through a carpule/cartridge system in a syringe. The material is somewhat 
25 condensed into place at 2-3 mm intervals, and light cured in between. Upon final filling, 
the restoration is polished and contoured using finishing burs (tips) on the handpiece (drill). 

If the tooth requires a lab fabricated restoration, such as an inlay, onlay or crown, 
further steps need to be taken (Inlay being a Class 2 restoration NOT including cusps, onlay 
being a Class 2 restoration including cusps, crown being full, or total coverage of the tooth). 
30 The tooth is shaped to make the final shape not have any undercuts, with walls as parallel as 
possible for retention purposes. 

Then an impression, or mold is taken of the tooth, which is in a material that 
remains accurate despite temperature changes, moisture, pouring stone into the mold and 
removing it several times. An impression of the opposite arch of teeth, or opposing arch, is 
35 taken also so that the technician can articulate, or put together the two arches and simulate 
the patient's mouth or bite. A registration of such a bite can be taken also and sent with the 
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case. So that the things sent to the lab for such a case are: impression of the tooth to be 
restored and adjacent teeth, model or impression of opposing teeth, and possibly a bite 
registration. 

Those skilled in the art should appreciate that the invention to determine the 
5 appropriate color shades of the tooth as illustrated in FIG. 14 can be accomplished by the 
methods herein, and/or by systems disclosed in the several figures. Using a wand of the 
invention, furthermore, various teeth (as in FIG; 14) can be acquired for digital evaluation. 
In accord with the invention, digital files of patients' teeth can be stored in memory of 
computer 14, FIG. 1, for a permanent record. A patient can then be evaluated over time for 
10 color changes. 

MISCELLANEOUS ASPECTS 

Although the VITA™ Shade guide is often discussed herein, it should be apparent 
that other shade guides and porcelains can be stored as REFERENCE IMAGES and 

15 compared to REAL IMAGES in alternative embodiments of this invention. Computer 
memory can store a large number of images, even from different manufacturers, so as to 
provide the optimum color fit to a patient's tooth. By way of example, IVOCLAR has one 
suitable shade guide, as well as various materials of porcelains, ceromers, polymers, and 
others. In accord with the invention, a database can store REFERENCE IMAGES for 

20 match correlation to REAL IMAGES. Alternatively, in one aspect, the invention performs 
a conversion to other manufacturer shades and or porcelains so that alternative laboratories 
can be used without undue concern for manufacturer alliance. Specifically, in accord with 
one aspect of the invention, a conversion between known shade guides is provided for 
increased lab selectivity. It will be appreciated that the conversion of digital images 

25 involves mapping from one set of color coordinates to another, which procedure is well 
known in the art and need not be considered in further detail. 

As described, one major problem due to auto brightness is that if there is not enough 
light material in the image, the auto brightness turns the gain up too high, and the R and G 
values saturate in the sample. By adding sufficient light colored reference area, the balance 

30 of light to dark is better, but the reference can saturate in places. This can be compensated 
some by using an end plate at the end of the sleeve that will be all white but with the black 
border round the sample aperture. A reference color area can be included in one corner so 
that the camera adjusts brightness for the constant white area leaving the reference and 
sample somewhere in the middle of the operable RGB ranges. 

35 . In still another aspect, instead of subtracting the RGB from the reference sample, an 

average of the reference sample is used. Specifically, over the range of images taken, an 
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average REF RGB (denoted "AVE REF RGB") is determined for the reference sample. For 
each individual image, the difference is calculated between the REF RGB and the AVE 
REF RGB. This delta RGB is then added to the image RGB to correct for any 
compensation in the camera. Thus if the image is brighter than the average image, the 

5 difference is subtracted from the sample values and vice versa. 

In still another aspect, the reference sleeve has an end plate which contains all tooth 
shades for a given manufacturer (so that one sleeve is used for a particular manufacturer). 
Any image therefore acquires the sample tooth as well as all the tooth shades; and image 
processing commences on the one digital image. This is advantageous in that camera color 

10 drift is compensated for since all images are taken at the same time. 

When determining tooth color, if the shade does not correspond to an existing shade 
(such as Al, D4), a system 400 is provided as shown in Fig. 15 which will create a recipe 
for whatever shade the tooth is. Color digital information about the tooth under test is sent 
to system 400 over a communication line such as the Internet; and system 400 selects 

15 porcelain mixtures from supply 402 to be sent and mixed within mixing subsystem 404. 
Subsystem 404 dispenses the mixture into a capsule 406 for the dentist, which administers 
the final mixture to the patient's tooth. 

The new shade material in carpule 406 is either provided chair-side, or reported to 
the technician over the Internet 410 (in which case, system 400 can be present at the 

20 technician's site). If for example the dentist needs to use a composite material like 

HELIOMOLAR by Ivoclar, and the desired shade is something between C2 and C3; the 
system determines the new shade of "C2/C3", mixes component composite materials 
accordingly, and dispenses to the dentist a premixed carpule/cartridge 406 of material to be 
used on that patient. Information to be sent to the lab can include a recipe that will instruct 

25 the technician as to how to create the new shade using whatever material is selected - 
porcelain, ceromer, pressed porcelain, etc. 

Fig. 16 illustrates in a block-diagram form the configuration of the interactive 
network system in a preferred embodiment. As shown, this system is implemented as a 
(preferably website) server 1610, having access to one or more databases 1630 containing 

30 information about different aspects of dental restoration. As noted, this information in a 
preferred embodiment includes materials, procedures and preparation recipes related to 
various dental restoration prostheses, and may include preparation design, recommended 
burs to achieve such a preparation, recommended temporization materials, cements that 
should be used with that given material, instructions on how to use such a cement and 

35 where to buy such materials, and others. In addition, database 1630 may contain 

information about different dental procedures, and in particular answers to commonly asked 
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questions. In yet another embodiment, the database may also contain patient histories, so 
that a dentist (or a technician) having access to the system can read the patient history 
regardless of where they are located physically, provided that they have proper 
authorization to access this information. The database may further be connected to external 

5 information source(s) providing input from product manufacturers, market research or 
others. In accordance with the present invention there is no restriction on the type of 
database 50 design that can be used. 

In operation, users can access the service over a communications network 1620, 
such as the Internet, using data entry devices, designated 1640 in Fig. 16. Generally, 

10 devices 1640 can be implemented as any color-display device capable of communicating 
over the network 1620. In the preferred embodiment where the network is the Internet, for 
example, such capability is provided by browsers, as known in the art. Thus, a device 1640 
can be a conventional personal computer 14, as illustrated in Fig. 1 . In alternative 
embodiments that do not communicate over the Internet such a device should have a 

15 capability to communicate over a communications network, e.g., an intranet, as known in 
the art. 

In a preferred embodiment, the server 1610 is a computer, the type of which may 
range from a personal computer to a workstation or a larger computer or a network of 
computers. The choice of a specific computer system is based on known trade-offs, and 

20 thus will not be discussed in further detail. In the preferred embodiment, the server is 
Internet-enabled but it will be appreciated that in alternative embodiments it can support 
another network, e.g., an intranet. Broadly, the server 1610 comprises a processor 1650 and 
random access memory (RAM) and read-only memory (ROM), collectively designated 
1670. The processor 1650 is adapted to execute instructions in different computer 

25 languages and can operate in different operating system environments, such as the 
PC/Windows™ environment. In a preferred embodiment, the server also has a data 
formatting device 1660 serving to provide interface with the database(s) 1630. In various 
embodiments the server can be provided with additional devices (not shown) such as a 
point-and-click device (like a mouse), keyboard, a monitor and various other devices, which 

30 may include, for example a printer 46. Preferably, the server is equipped with means for 
connecting to external communications media, and may include various connectivity types, 
such as a LAN connection. 

It will be appreciated that using the system shown in Fig. 16, different users 1640 
can access information stored at the network server (or to which the server has access). It 

35 will also be appreciated that the system would enable direct communication between users, 
such as on-line communication between a dentist and a dental restoration laboratory, where 
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the two sides can simultaneously have access to a patient record, an image of tooth in need 
for restoration or others. Naturally, if desired, such communications can be established over 
dedicated lines although in accordance with this invention it is desirable to use open 
systems, such as the Internet because of the flexibility they provide. Further, while they are 

5 shown separate for illustration purposes, it should be apparent that the server 1610 can 
physically be located at dentist's (or laboratory) office. Communications networks and 
servers of the type illustrated in Fig. 16 are generally known in the art and thus need not be 
described with specificity. 

While the invention has been described with reference to the preferred 

10 embodiments, it will be appreciated by those of ordinary skill in the art that various 

modifications can be made to the structure, form and operation of the various embodiments 
of the invention without departing from its spirit and scope, which is defined in the 
following claims. 

15 APPENDIX A contains, for disclosure purposes, non-limiting source code for use 

with certain aspects of this invention. 
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THE CLAIMS 

What is claimed is: 

5 1 . An interactive dental restoration method between a dentist and a dental 

restoration laboratory which comprises: 

identifying a dental restoration need in a patient; 
designing a preliminary treatment plan that includes design criteria for 
preparation of a dental prosthesis to be placed in the patient to satisfy the dental restoration 
10 need; 

transmitting the preliminary treatment plan via a communications network to 
a dental restoration laboratory; and 

communicating a final treatment plan, including modifications to the 
preliminary treatment plan if necessary, to the dentist. 

15 

2. The method of claim 1 wherein the dentist prepares the preliminary 
treatment plan and the design criteria include digital image representations of the dental 
restoration need. 

20 3. The method of claim 2 which further comprises evaluating the preliminary 

treatment plan at the laboratory before communicating the final treatment plan to the 
dentist. 

4. The method of claim 3 which further comprises implementing the final 
25 treatment plan in the patient and transmitting interim preparation information to the 

laboratory for survey with confirmation prior to completing the final treatment plan. 

5. The method of claim 3 when the step of transmitting and evaluating the plan 
are codirected over the communications network. 

30 

6. The method of claim 5 which further comprises preparing a dental prosthesis 
that satisfies the design criteria of the final treatment plan and placing the prosthesis in a 
patient. 

35 
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7. The method of claim 6 which further comprises verifying that the dental 
prosthesis is prepared according to the final treatment plan prior to placement of the dental 
prosthesis in the patient. 

5 8. The method of claim 6 wherein one of the design criteria or the 

modifications include proposed decay excavation, tooth preparation, or prosthesis color. 

9. The method of claim 6 wherein the digital image representations include 
REAL IMAGE and REFERENCE IMAGES and the modifications include correlation of a 

10 color selection for the prosthesis to match the REAL IMAGE. 

10. The method of claim 4 wherein the design criteria include tooth preparation 
and proposed decay excavation, and which further comprises communication, a 
confirmation or modification, from the laboratory, of the acceptability of one or more of the 

15 proposed design criteria. 

11. A computer-based dental restoration system comprising: 

a network server having a database storing information about materials, 
procedxires and preparations concerning dental restoration prostheses; 
20 a communications network providing access to said network server; and 

one or more computers for accessing information stored at the database over 
the communications network and displaying the information in a humanly readable format. 

12. The dental restoration system of claiml 1 wherein the computer(s) can be 
25 located at a dental office and the communications network is the Internet. 

13. The dental restoration system of claim 1 1 wherein information stored in the 
database comprises preparation diagrams, reduction dimensions, margin design and burs for 
specific dental restoration prostheses. 

30 

14. The dental restoration system of claim 1 1 wherein the database further stores 
information concerning one or more patients having dental restoration needs. 

15. The dental restoration system of claim 1 1 wherein the network server further 
35 comprises application programs for enabling users to query the database regarding specific 
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materials or procedures concerning dental restoration prostheses for confirmation, 
verification, modification or evaluation of the same. 

16. The dental restoration system of claim 15 wherein the one or more 

5 computers at the dental office receive answers from the database to said queries and further 
comprising at least one printer located at the dental office to print said answers for reference 
in carrying out the treatment plan. 

17. The dental restoration system of claim 1 1 further comprising at least one 
10 computer at a dental restoration laboratory, said at least one computer having access to the 

network server and said or more computers at the dental office over the communications 
network. 

18. The dental restoration system of claim 1 1 further comprising a digital camera 
15 for taking digital images of patient's teeth in need of dental restoration and a 

communication link for transmitting the digital images to the one or more computers at the 
dental office. 

19. The dental restoration system of claim 1 8 wherein the one or more 

20 computers at the dental office stores the digital images of patient's teeth in need of dental 
restoration and the communications network forwards the digital images to the database for 
storage therein. 

25 
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A ppendix A 
Software Modules 



Number of 
Module Title Pages 



About 1 page 

ImageDisplay 3 pages 

RefDisplay 4 pages 

SampleLocator 4 pages 

Sdimain 8 pages 

ShadeData 8 pages 

SplashScreen 3 pages 

Tooth Object 6 pages 



WO 00/25696 0 PCT/US99/22857 



unit About; 
interface 

uses. Windows, Classes, Graphics, Forms, Controls, S 
Buttons,. ExtCtrls; 

type 

JTAboutBox - class (TForm) 

Panel 1: TPanel; 

OKButton: TButton; 

Programlcon : TImage; 

ProductName: TLabel; 

Version: TLabel; 

Copyright: TLabel; 

Comments: TLabel; 
private 

{ Private declarations } 

public r 
{ Public declarations } 

end; 
var 

AboutBox: TAboutBox; 
implementation 
{$R * . DFM) 
end. 
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with pmilmage .Canvas do 
begin 

moveto{Left, Top) ; 

lineto(Left, Bottom); 

lineto (Right, Bottom); 

lineto(Right, Top); 

lineto (Left, Top) ; 

if Rows > 1 then 

begin - 

Spacing := (Bottom - Top) / Rows; 

for index 1 to Rows - 1 do 

moveto (Left+1, Top + trunc (Spacing * index)); 
lineto (Right-1 # Top + trunc ( Spacing * index)); 
end; 
end; 

if Columns > 1 then 
begin- 

Spacing (Right - left) / Columns; 
for index 1 to Columns - 1 do 
begin - 

moveto (Left + trunc (Spacing * index), Top+1) ; 
lineto (Left + trunc (Spacing * index), Bottom-1); 
end; 
end; 
end; 

end; // DrawGrid 

procedure Tf rmlmageDisplay . DrawGrids ; 
begin 

SetROP2 (pmilmage. Canvas . Handle, R2__NOT) ; 

if DisplayGrid then 
begin 

DrawGrid (Ref A, RefR, RefC) ; 
DrawGrid (SampleA, SampleR, SampleC) ; 
end; 

end; // DrawGrids 

procedure Tf rmlmageDisplay . FormCreate (Sender: TObject ) ; 
begin 

DisplayGrid := false; 
end; 

procedure Tf rmlmageDisplay . DefineGrids (Ref Area: TRect; 

RefRows : integer; 
Ref Cols : integer; 
SampleArea : TRect; 
SampleRows : integer; 
SampleCols : integer) ; 

begin 

Ref A Ref Area; 
RefR : = RefRows; 
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S gAnalysis.Cells[GreenMaxColumn, Rowlnsertlndex] := FloatToStrF (GreenMax, 

ff Fixed, Precision, Digits); 

sgAnalysis. Ceils [ BlueMaxColumn, Rowlnsertlndex) FloatToStrF (BlueMax, 

ffFixed, Precision, Digits) ; 

sgAnalysis. Cells [RedMinColumn, Rowlnsertlndex] := FloatToStrF (RedMm, 

ffFixed, Precision, Digits); 

sgAnalysis. Cells [GreenMinColumn, Rowlnsertlndex) FloatToStrF (GreenMm, 

ffFixed, Precision, Digits); ' , 

sgAnalysis. Cells (BlueMinColumn, Rowlnsertlndex] := FloatToStrF (BlueMm, 

ffFixed, Precision, Digits);. 

Variation (RedMax - RedMin) + (GreenMax -..GreenMm) + (BlueMax - 

^^sgAnalysis. Cells [ VariationColumn, Rowlnsertlndex] := FloatToStrF (Variation 
ffFixed, Precision, Digits) ; 
end; 

inc (Rowlnsertlndex) ; 
end; 

procedure Tf rmRef erenceDisplay . ShowShades ; 
var 

Shadelndex : integer; 
CurrentShade : TShadeColours ; 
begin 

Rowlnsertlndex := TitleRow + 1; 

for Shadelndex : « 0 to DisplayShades . ShadeList . Count - 1 do 
begin 

CurrentShade := DisplayShades . ShadeList . I terns [ Shadelndex] ; 
InsertShadeData (CurrentShade) ; 
end; 
end; 

procedure Tf rmRef erenceDisplay. LoadShades (Shades : TShadeRef erences ) ; 
var 

IDisplayRow : integer; 
begin 

{ First Clear Old list } 

if DisplayShades. ShadeList .Count > 0 then 

for IDisplayRow := 1 to DisplayShades . ShadeList . Count do 
sgAnalysis .Rows [IDisplayRow] .Clear ; 

//DisplayShades . Free; 
DisplayShades := Shades; 

sgAnalysis . RowCount :« Shades . ShadeList . Count + 1; 
ShowShades; 
end; 

procedure Tf rmRef erenceDisplay . edGridColChange ( Sender : TObject) ; 
begin 

if Visible then 

begin 

DisplayColumn := StrToInt (edGridCol . Text ) ; 
ShowShades; 
end; 
end; 

procedure Tf rmRef erenceDisplay . edGridRowChange (Sender : TObject) ; 
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unit ImageDisplay; 

interface 

us e s 

Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs, 
TMultiP, ExtCtrls; 

type 

Tf rmlmageDisplay » class (TForm) 
pnllmage: TPanel; 
pmilmage: TPMult ilmage ; 
procedure FormCreate (Sender : TObject); 
procedure pmilmage Paint ( Sender : TObject); 

private 

{ Private declarations } 
RefA : TRect; 
RefR : integer; 
RefC : integer; 
SampleA : TRect; 
SampleR : integer; 
SampleC : integer; 
DisplayGrid : boolean; 

procedure DrawGrid { Area : TRect; Rows, Columns : integer); 
procedure DrawGrids; 
public 

{ Public declarations } 

procedure Def ineGrids { Ref Area : TRect; 

RefRows : integer; 

RefCols : integer; 

SampleArea : TRect; 

SampleRows : integer; 

SampleCols : integer) ; 

procedure HideGrid; 
end; 

var 

f rmlmageDisplay: Tf rmlmageDisplay; 
implementation 
($R * . DFM} 

procedure Tf rmlmageDisplay . DrawGrid (Area : TRect; Rows, Columns : integer); 
var 

Spacing : real; 
index : integer; 
ScaleX : real; 
ScaleY : real; 

Left, Right, Top, Bottom : integer; 
begin 

ScaleX :« pmilmage .Width/ 64 0 ; 
. ScaleY := pmilmage . Height /4 80 ; 
L e ft := Round (Area . Left * ScaleX); . 
Right := Round ( Area . Right * ScaleX); 
Top :» Round {Area . Top * ScaleY)'; 
Bottom := Round (Area . Bottom * ScaleY); 
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RefC := RefCols; 
SampleA : = SampleArea; 
SampleR : « SampleRows ; 
SampleC :« SampleCols; 
DisplayGrid := true; 
pnvi Image . Repaint ; 
end; 

procedure Tf rmlmageDisplay . HideGrid; 
begin 

"DisplayGrid := false; 
Repaint; 
end; , 

procedure Tf rmlmageDisplay . pmilmagePaint (Sender: TObject )~; 
begin . 

DrawGrids; 
end; 

end. 
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NewCalibration false; 

"iniFile TIniFile .Create (DiskDrive + 'AnalyseV + IniFilename) ; 
DefaultShadeFilename := IniFile . ReadString ( IniShadeSetSection, 

IniDefaultGuide, 'ERROR 1 ); vi ^ . , ^ 

DefaultShadeFilename := DiskDrive + 'AnalyseV + DefaultShadeFilename; 

LoadShadeSet (DefaultShadeFilename) ; 

finally 

IniFile. Free; 

end; 
end; 

function TfrmShadeAnalyzer.Analyselmage (FileName : string; ShadeName : string) 

TShadeColours; 

var 

ShadeColours : TShadeColours; , 
Tooth : TTooth; . 
DeltaRed : real; 
DeltaGreen : real; 
DeltaBlue : real; 
PixelPercent : real; 
Ref Area : TRect; 
SampleArea : TRect; 
begin 

Tooth := TTooth. Create; 

{ Analyse The Reference Area } 

f rmlmageDisplay . HideGrid; 

Tooth. LoadBitmapFromFile (FileName) ; 

f rmlmageDisplay . pmi Image . Picture . Bitmap . Assign (Tooth . ToothBi tmap) ; 
Application. ProcessMes sages ; 

Ref Area : - Tooth. FillSearchSampleLimits ( f rmSampleLocator . Ref erenceLocation) 
frmlmageDisplay. DefineGrids(RefArea, RefRows, RefColumns, Rect ( 0 , 0 , 0 , 0 ) , 
SampleRows, SampleColumns) ; 
Application. ProcessMes sages ; 

Tooth. RemoveMask (Ref Area) ; 

frmlmageDisplay. pmi Image. Picture. Bitmap. Assign (Tooth . ToothBitmap) ; 
Application. ProcessMessages ; 

Tooth. Analysed, 0/ RefArea, RefRows, RefColumns, DeltaRed, DeltaGreen, 
DeltaBlue, PixelPercent) ; 

DeltaRed : = RefRedMedian - DeltaRed; 
DeltaGreen :» Ref GreenMedian - DeltaGreen; 
DeltaBlue := Ref BlueMedian - DeltaBlue; 

{ Now Analyse the Sample Area } - 

frmlmageDisplay. HideGrid; 

Tooth. LoadBitmapFromFile (FileName) ; 

frmlmageDisplay . pmi Image . Picture . Bitmap . Assign (Tooth . ToothBitmap) ; 
Application . ProcessMessages ; 

SampleArea Tooth . FillSearchSampleLimits ( f rmSampleLocator . SampleLocat ion) 
frmlmageDisplay. DefineGr ids ( Rect .,(.0 , 0,0,0), RefRows, RefColumns, SampleArea, 
SampleRows, SampleColumns); 
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Application. ProcessMessages; 

Tooth. RemoveReflect ion (SampleArea) ; . 

f rmlmageDisplay.. pmilmage . Picture . Bitmap . Ass ign (Tooth . ToothBitmap) ; 

Application . ProcessMessages ; 

ShadeColours Tooth .AnalyseGrid ( SampleArea , S.ampleRows, Sample Columns , 

DeltaRed, DeltaGreen, DeltaBlue) ; 

ShadeColours. Name := ShadeName; 
Result ShadeColours; 
Tooth. Free; 
end; 

procedure Tf rmShadeAnalyzer . CalibrateClick ( Sender : TObject); 
var 

FilePath : string; 
Filelndex : integer; 
IFilename : string; 
ShadeName : string; 
ProgressBar : TProgressBar ; 
ShadeColours : TShadeColours; 
begin 

. OpenDialog. Title 'Files To Analyse'; 
OpenDialog.InitialDir :« DiskDrive + ' Analyse\Pictures \ ; 
OpenDialog. DefaultExt := GraphicExtension (TBitmap) ; 

OpenDialog. Filter := GraphicFil ter (TBitmap) ; , ■ w fcf= . . 

OpenDialog. Options : - ' [of AllowMultiSelect, of PathMus tExist , of FileMus tExist ] ; 
if OpenDialog . Execute then 

with OpenDialog . Files do 

begin 

edShadeSetName.Text := 'New Calibration'; 
NewCalibration true; 

StatusBar. SimpleText 'Loading Calibration Bitmaps ; 

Shades. Free; # , 

Shades : = TShadeRef erences . Create ; // we will build a new list 

FilePath := Extract FilePath (OpenDialog . FileName) ; 
ProgressBar := TProgressBar . Create { self ) ; 
ProgressBar . Parent :« self; 
ProgressBar .Align := alBottom; 
ProgressBar . Min : = 0; 
ProgressBar .Max :~ Count+2; 

ProgressBar .Step := 1; // the amount to move with the Steplt method 

for Filelndex : e 0 to Count - 1 dp 

begin 

IFileName : - Strings [Filelndex] ; 

{ Get the Shade Name from the filename }. 
ShadeName : = ExtractFilename (IFilename) ;' 

StatusBar .SimpleText := 'Loading Calibration Bitmap : • +ShadeName ; 
ShadeName :*= Uppercase (Copy (ShadeName, 1, Pos ( ' . \ ShadeName) - 2)>; / 
remove the letter 

ShadeColours := Analyselmage ( 1 FileName , ShadeName); 
Shades . AddSample (ShadeColours) ; 
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ProgressBar.Stepit; // Move one Step amount 
end; 

Application . ProcessMess ages ; 

{ Get the Shades into alphabetical order } 
StatusBar .SimpleText :- 'Sorting Shade Samples'; 
Shades . SortList; 

ProgressBar.Stepit; // Move one Step amount 

{ Process the Shades Data to get average sets > 

StatusBar. SimpleText := ' Reducing Shades to Reference Set ■ ; 

Shades . ReduceList ; 

ProgressBar.Stepit; // Move one Step amount 
" Shades. SortList ; 
ProgressBar . Free; 

StatusBar . SimpleText := 'Done'; 
end; 

end; 

procedure Tf rmShadeAnalyzer . ShowImagelClick (Sender : TObject); 
begin 

f rmlmageDisplay . Show; 
end; 

function TfrmShadeAnalyzer.FindNearestShade (Sample : tShadeColours) : string; 
var 

. CurrentShade -..TShadeColours; 
ShadeName : string; 
ShadeDif f erence : real; 
CurrentDif ference : real; 
Shadelndex : integer; 
begin 

ShadeDif ference 10000C0; 
ShadeName : = 'None 1 ; 

for Shadelndex : - 0 to Shades . ShadeList . Count - 1 do 
begin 

CurrentShade := Shades . ShadeList . I terns [Shadelndex] ; 
CurrentDif ference CurrentShade . Colour Difference (Sample ) ; 
if CurrentDif ference < ShadeDi f ference then 
begin 

ShadeName : « CurrentShade . Name ; 
ShadeDif ference := CurrentDi f ference ; 
end; 
end; 

result := ShadeName; 
end; 

procedure Tf rmShadeAnalyzer . ShowRef erencelCl ick ( Sender : TObject) ; 
begin 

f rmRef erenceDisplay . LoadShades (Shades) ; 
f rmRef erenceDi splay .ShowModal ; 
end; 
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procedure Tf rmShadeAnalyzer . SetSampleLoclClick (Sender : TObject) ; 
begin 

f rmSampleLocator . ShowModal ; 
end; 

procedure Tf rmShadeAnalyzer .AnalyselClick < Sender : TObject); 
var 

lFilename : string; 
SampleColours : TShadeColours ; 
ProgressBar : TProgressBar; 
begin 

OpenDialog. Title := 'Files To Analyse'; 

OpenDialog.InitialDir := DiskDrive + ' Analyse\Pictures\ ; 

OpenDialog. DefaultExt := GraphicExtension (TBitmap) ; 

OpenDialog. Filter GraphicFilter (TBitmap) ; 

OpenDialog. Options := (of PathMustExist , of FileMustExxs t ] ; 

if OpenDialog. Execute then 

begin 

edNearest .Text : .« ' 1 ; 

StatusBar .SimpleText : = 'Analyzing Sample'; 

ProgressBar := TProgressBar . Create (self ) ; 
ProgressBar . Parent := self; 
ProgressBar .Align := alBottom; 
ProgressBar. Min ":=* 0; 

ProgressBar. Max :» 3; . , _ , 

ProgressBar. Step := 1; // the amount to move with the Steplt method 

lFilename : = OpenDialog . Filename; 

ProgressBar . Steplt ; 
Application . ProcessMes sages ; 

SampleColours : = Analyselmage ( lFileName , 'Unknown'); . 

ProgressBar . Steplt ; 
Application . ProcessMes sages ; 

StatusBar. SimpleText := 'Searching for Nearest Shade'; 
edNearest .Text := FindNearestShade (SampleColours ) ; 

ProgressBar . Steplt; 
Application . ProcessMessages; 

StatusBar . SimpleText : = 'Done'; 
ProgressBar . Free ; 
end; 
end; 

procedure Tf rmShadeAnalyzer . btnSaveClick (Sender : TObject) ; 
var 

lFilename : string; 
OutStream : TFileStream; 
IniFile : TIniFile; 
begin ( 
SaveDialog. Title : = 'Shade Guide Filename to Save ; 
SaveDialog . InitialDir := DiskDrive + 'AnalyseV; 
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SaveDialog. DefaultExt := 1 SDS'; 
SaveDialog. Filter := ' Shade -Guides I *. SDS ' ; 
SaveDialog. Options := ( of PathMustExist ] ; 
if SaveDialog . Execute then 

with SaveDialog do 

begin 

if not FileExists (Filename) or 

(MessageDlg (Format ( 'Overwrite %s? ' , [ ExtractFilename (Filename) ] ) , 
reconfirmation, [mbYes, mbNo], 0) - mrYes) then 

begin 
try 

OutStream :<= TFileStream . Create ( Filename , fmCreate or fmShareCompat); 
Shades . SaveToStream (OutStream) ; 

lFilename ExtractFilename ( SaveDialog . Filename ) ; 
edShadeSetName.Text copy ( 1 Filename, 1 , Length ( lFilename) - 4); 
NewCalibration := false; 
try 

IniFile : = TIni File . Create ( Dis kDrive + 'AnalyseV + IniFilename) ; 
IniFile.WriteString ( IniShadeSetSection, IniDef aultGuide , lFilename) 
finally 

IniFile . Free; 
end; 
finally 

OutStream. Free; 
end; 
end; 
end; 

end; 

procedure Tf rmShadeAnalyzer . LoadShadeSet ( Filename : string); 
var 

InStream : TFileStream; 
IniFile : TIniFile; 
lFilename string; 
begin 
try 

edShadeSetName.Text := 'Loading... ; 

InStream := TFileStream. Create ( Filename , fmOpenRead or fmShareCompat); 
Shades. Free; 

Shades := TShadeRef erences . Create; // we will build a new list 
Shades . LoadFromStream ( InStream) ; 
lFilename : = Extract Filename ( Filename) ; 

edShadeSetName.Text :~ copy ( lFilename, 1 , Length (lFilename) - 4); 
try 

IniFile TIniFile . Create ( DiskDrive + 'AnalyseV + IniFilename) ; 
IniFile.WriteString ( IniShadeSetSection, IniDef aultGuide, lFilename) ; 

finally 

IniFile. Free; 
end; 
finally 

InStream. Free; 

end; 
end ; 

procedure Tf rmShadeAnalyzer . btnLoadClick ( Sender : TObject); 
begin 

OpenDialog .Title := 'Shade Guide Set to Load*; 
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OpenDialog. InitialDir DiskDrive + 'AnalyseV; 
OpenDialog. DefaultExt := 'SOS 1 ; 
OpenDialog. Filter :« 'Shade Guides I * . SDS * * 

OpenDialog. Options := [ of PathMustExist , of FileMustExxst ] , 
if OpenDialog. Execute then 

LoadShadeSet (OpenDialog. Filename) ; 

end; ' * . 

procedure Tf rmShadeAnalyzer . FormClose (Sender : TObject; 
var Action: TCloseAction) ; 

^rciosing Program - Check for unsaved; Calibration Set } 
if (NewCalibration) and 

(MessageDlg( 'Calibration Load Not Saved Save Now? 

mtConfirmation, [mbYes, mbNo],.0> - mrYes) then 
btnSaveClick (self) ; 

end; 

procedure Tf rmShadeAnalyzer . FormActivate ( Sender : TObject); 

begin 

if Startup then 
begin 

Startup := false; 

f rmSplashScreen. Show; 

Application. ProcessMessages ; 

{ $1 FDEF SLIDELOGO) 

frmSplashScreen.Timerl . Interval := 1000; 

{ $ELSE } 

f rmSplashScreen. Timerl . Interval :« 3000; 
{ $ENDI F} 

end; 
end; 

end. 



Sdimain - page 8 



WO 00/25696 PCT/US99/22857 

13 



unit SplashScreen; 
interface 

^Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs, 
ExtCtrls, TMultiP; 

ty TSplashState = (splCenter, splMoving, splDone) ; 

TfrmSplashScreen - class (TForm) 
Timerl: TTimer; 
pnlLogo; TPanel; 
Imagel: TImage; 

procedure TimerlTimer (Sender : TObject); 
procedure FormCreate ( Sender : TObject); 
procedure -FormPaint (Sender : TObject); 

private 

{ Private declarations } 

SplashState : TSplashStat e ; 

StartPosition : TPoint; 

StartSize : TPoint; 

VirticalStep : integer; 

HorizontalStep : integer; 

widthStep : integer; 

HeightStep : integer; 

CanPaint : boolean; 
public 

{ Public declarations ) 

end; 
var 

f rmSplashScreen: TfrmSplashScreen; 
implementation 
{$R * . DFM} 
const 

FinishPositidn : TPoint - (x:580; y:700); 
FinishSize : TPoint = (x: 430; y:50);" 

Duration - 2000; // ms 

Movelnterval ~ 50; / / ms 

procedure Tf rmSplashScreen . Timer ITimer ( Sender : TObject); 
begin 

{ $IFDEF SLIDELOGO} ■ 
if abs(FinishPosition.x - Left) < abs (HorizontalStep) then 

HorizontalStep :« HorizontalStep div abs (HorizontalStep) ; 
if abs(FinishPosition.Y - Top) <, abs,( VirticalStep) then 
VirticalStep := VirticalStep div abs (VirticalStep) ; 

if abs (FinishSize. x - Width) < abs (WidthStep ) then 

WidthStep WidthStep div abs (WidthStep) ; 
if abs (FinishSize.Y - Height) < abs ( HeightStep) then 
HeightStep := HeightStep div abs ( Height Step ) ; 
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CanPaint :- false; 

case SplashState of 
splCenter: 
begin 

Timerl - Interval Movelnterval; 
SplashState := splMoving; 
end; 
splMoving: 

begin , 
if (Top = FinishPosition.y) and (Left = FinishPosition . x) and 
(Height - FinishSize . y) and (Width - FinishSize . x) then 
SplashState :» splDone 
else 
begin 

if Top <>. FinishPosition. y then 

Top :« Top + VirticalStep; 
if Left <> FinishPosition . x then 

Left := Left + HorizontalStep; 
if Height <> FinishSize. y then 

Height := Height + HeightStep; 
if Width <> FinishSize. x then,. : 

Width :» Width + WidthStep; 

end; 
end; 
splDone : 
begin 

Timerl . Enabled :~ false; 
end; 

end; 

CanPaint := true; 
{ $ELSE } 

Height :« FinishSi ze . y ; 

Width := FinishSize .x; 

Top FinishPosition.y; 

Left := FinishPosition. x; 

pnlLogo . BevelWidth := 2; 
{ $ENDIF} 
end; 

procedure Tf rmSplashScreen . FormCreate (Sender : TObject) ; 
begin 

SplashState := splCenter; 
StartPosition . x Left; 
StartPosition.y :« Top; 
StartSize.x :«= Width; 
StartSize.y := Height; 
//VirticalStep := (FinishPosition.y - StartPosition.y) div (Duration div 

Movelnterval) ; 

//HorizontalStep :« (FinishPosition . x - Start Position . x) div (Duration div 
Movelnterval) ; 

//HeightStep ( FinishSize . y - StartSize.y.) div (Duration div Movelnterval) 
//WidthStep := < FinishSize . x - Starts i.ze . x ) div (Duration div Movelnterval); 

VirticalStep : = 1; 

HorizontalStep := 1; 

HeightStep := -1; 
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WidthStep := -1; 
CanPaint := true; 
end; 

procedure Tf rmSplashScreen .. FormPaint ( Sender : TObject); 
begin 

if CanPaint then 
Imagel. Repaint; 

end; 
end. 
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unit ToothObject; 



interface 
uses 

Windows, SysUtils, Classes, Graphics, 
ShadeData; 

type 

TTooth = class (TObject) 
private 

Ref erencelnitialised : boolean; 

function CalculateTestArea (Row, Col : integer; 

Area : TRect; 

NoRows, NoCols : integer) : TRect; 

public 

Red : real; 

Green : real; 

Blue : real; 

Hue : real; 

Saturation : real; 

Intensity : real; 

Ref Red : real; 

Ref Green : real; 

RefBlue : real; 

RefHue : real; 

Ref Saturation : real; 

Reflntensity : real; 

ToothBitmap : TBitmap; 

ToothBitmapMask : TBitmap; 

constructor Create; 

procedure Free;' 

procedure LoadBitmapFromFile ( Filename : String) ; 
procedure RemoveRef lection (TestArea : TRect); 
procedure RemoveMask (TestArea . :' TRect ) ; 

function FillSearchSampleLimit s (StartPoint : TPoint) : TRect; 

procedure Analyse (Row, Col : integers- 
Area : TRect; NoRows, NoCols : integer; 
var R, G, B : real;^ 
var PixelPercentage : real); 

function AnalyseGrid ( Area : TRect ; NoRows , NoCols : integer; 

DeltaRed, DeltaGreen, DeltaBlue : real) : 

TShadeColours; 

procedure CalculateHSI (R, G, B : real; var Hue, Sat, Int : real); 

ends- 



implementation 

uses 

Dialogs, Math; 

const 

RedMask : longint = $OOOOO0FF; 

GreenMask : longint = $0000FF00; 
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BlueMask ; longint « $O0FF0000; 

Boundrylntensity ■* 0.58; // sum of RGB 

Ref lectionlntensity =■ 1.86; 

type 

TDirection = (Up, Down, Left, Right); 

constructor TTooth . Create; 
begin 

ToothBitmap := TBitmap .Create; 
ToothBitmap. Width 64 0; 

ToothBitmap. Height : » 480; 
ToothBitmapMask TBitmap . Create ; 
ToothBitmapMask. Width :« 64 0; 
ToothBitmapMask. Height :« 480; 
Ref erencelnit iaiised := false; 

end; 



procedure TTooth. Free; * 
begin 

ToothBitmap. Free; 

ToothBitmapMas k . Free ; 

Inherited Free; 
end; 

function TTooth. FillSearchSampleLimits (StartPoint : TPoint) : TRect; 
procedure FillSearch (StartpointX, StartpointY : integer) ; 
var 

Red, Green, Blue : integer; 
x, y : integer; 
begin 

if StartPointX > Result. Right then 

Result. Right :~ StartpointX; 
if StartpointX < Result. Left then 

Result. Left :« StartpointX; 
if StartpointY > Result . Bottom then 

Result . Bottom := StartpointY; 
if StartpointY < Result. Top then 

Result. Top := StartpointY; 

for x := -1 to 1 do 
for y :- -1 to 1 do 

if (StartpointX+x < 640) and ( StartpointX+x > 0) and 
(StartpointY+y < 480) and (StartpointY+y > 0) and 

(ToothBitmapMas k. Canvas. Pixels [ StartpointX+x, StartpointY+y] <> 0) 

then 

begin 

Red ToothBitmap . Canvas . Pixels [ StartpointX+x, StartpointY+y] and 

RedMask; 

Green : = (ToothBitmap . Canvas . Pixels [ S tartpointX+x, StartpointY+y] and 
GreenMask) shr 8; 

Blue : = (ToothBitmap . Canvas .Pixels [StartpointX+x, StartpointY+y) and 
BlueMask) shr 16; 

if ({Red + Green + Blue-)/255) > Boundrylntensity then 
begin 

ToothBitmapMask. Canvas. Pixels [ Star tpointX+x, StartpointY+y) :« 0; 
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FillSearch(StartpointX+x, StartpointY+y ) ; 
end 
end; 

end; 
begin 

Result := Rect(640, 480, 0, 0) ; 
FillSearch (St art Point .X, Start Point . Y) ; 

end; 

function TTooth . CalculateTestArea ( Row, Col : integers- 
Area : TRect; 

NoRows, NoCols : integer) : TRect; 

var 

TestArea : TRect; 
ColSpacing : real; 
RowSpacing : real; 
begin 

with Area do 
begin 

ColSpacing := (Right - Left) / NoCols; 
RowSpacing :« (Bottom - Top) / NoRows; 
TestArea. Top : •» Top + Trunc (RowSpacing * Row); 
TestArea. Bottom := Top + Trunc (RowSpacing * (Row + 1) ) ; 
TestArea. Left := Left + Trunc (ColSpacing * Col); 
TestArea. Right : - Left + Trunc (ColSpacing * (Col + 1)J; 
end; 

CalculateTestArea :« TestArea; 
end; // CalculateTestArea 

procedure TTooth. RemoveRef lection (TestArea : TRect); 
var 

i : integer; 
j : integer; 

Red, Green, Blue : integer; 
begin 

with . TestArea do 

for i := Left to Right do 

for j := Top to Bottom do 
. begin 

if (ToothBitMapMask. Canvas. Pixels(i,j] <> 0) then 

ToothBitMap. Canvas . Pixels [i, j 3 0 
else 
begin 

Red ToothBitMap . Canvas . Pixels [ i , j ] and RedMask; 

Green : « (ToothBitMap . Canvas . Pixels [ i , j ] and GreenMask) shr 8; 

Blue :« (ToothBitMap. Canvas . Pixels [i, j ] and BlueMask) shr 16; 

if ((Red + Green + Blue) / 255 > Ref lectionlnt ensity) then 
ToothBitMap .Canvas . Pixels [i, j ] : = 0; 

end; 

end; 

end; // RemoveRef lection 

procedure TTooth . RemoveMask (TestArea : TRect); 
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var 

i 



: integer; 
: integer; 



j 

begin 



with TestArea do 
for i Left to Right do 

for j := Top to Bottom do 

if (ToothBitMapMask. Canvas . Pixels [i, j ] <> 0) then 
ToothBitMap. Canvas . Pixels [i, j ] := 0; 



end; // RemoveRef lection 



procedure TTooth . Analyse ( Row, Col : integer; 



NoCols ; integer; 



Area : TRect; NoRows, 
var R, G, B : real; 
var PixelPercentage : 



real ) ; 



var 

TestArea : TRect; 
PixelCount : longint; 

i : integer; 
j : integer; 
RedTotal : longint; 
GreenTotal : longint; 
BlueTotal : longint; 
Red, Green, Blue : integers- 
begin 

TestArea : = CalculateTestArea (Row, Col, Area, NoRows, NoCols) ; 
with TestArea do 

PixelCount (Right - Left + 1) * (Bottom - Top + 1); 

// Now average ignoring reflections and blemishes 
RedTotal := 0; 
GreenTotal := 0; 
BlueTotal :» 0; 

with TestArea do 

for i Left to Right do 

for j : - Top to Bottom do 

begin 

if ToothBitMap. Canvas . Pixels [i, j ] <> 0 then 
begin 



Red : = ToothBitMap . Canvas . Pixels [ i , j ] and RedMask; 
Green := (ToothBitMap . Canvas . Pixels [ i, j ] and GreenMask) shr 8; 
Blue := (ToothBitMap. Canvas . Pixels [i, j ] and BlueMask) shr 16; 

RedTotal RedTotal + Red; 
GreenTotal : = GreenTotal + Green; 
BlueTotal :« BlueTotal + Blue; 



end 
else 
begin 



PixelCount := PixelCount - 1;. // Ignored this pixel 



end; 



end; 
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// Normalised RGB 

if PixelCount* > 0 then 

begin 

R RedTotal / PixelCount / 255; 
G := GreenTotal / PixelCount / 255; 
B BlueTotal / PixelCount / 255; 
with TestArea do 

PixelPercentage := PixelCount / ( (Bottom-Top+1 )* (Right - Left+1)); 

end 

else 

begin 

R : » 0 ; 

G : - 0 ; 

B :=* 0; 

PixelPercentage :- 0; 
end; 

end; // Analyse 

function TTooth . AnalyseGrid (Area : TRect; 

NoRows, NoCols : integer; 

DeltaRed, DeltaGreen, DeltaBlue : real) : 

TShadeColours; 
var 

Row, Col : integer; 

Red, Green, Blue, PixelPercent : real; 
begin 

Result := TShadeColours .Create ; 

for Row :« 0 to NoRows - 1 do 
for Col : s 0 to NoCols - 1 do 
begin 

Analyse (Row, Col, Area, NoRows v . NoCols , 

Red, Green, Blue,* PixelPercent); 
Result .GridColours [Col + 1, Row+1] : ~ TShadeColourElement . Create; 
Result .GridColours [Col+1, Row+ 1 ] . StoreColour ( Red + DeltaRed, 

Green + DeltaGreen, 



Blue + DeltaBlue, 
PixelPercent) ; 



end; 

end; 



procedure TTooth . CalculateHSI ( R, G, B : real; var Hue, Sat, Int : real); 

function Minimum (vl, v2, v3 : real) : real; 
begin 

if (vl <= v2) and (vl <= v3) then 
minimum : = vl 

else 

if (v2 <= vl) and (v2 <^ v3) then 
Minimum : - v2 
else 

Minimum : = v3; 

end; 

function Maximum (vl, v2, v3 : real) : real; 
begin 

if (vl >= v2) and (vl >= v3) then 
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Maximum vl 

else 

if (v2 >= vl) and (v2 >= v3) then 
Maximum : = v2 
else , 
Maximum v3; 

end; 
begin 

//Calculation using Gonzalez and Woods 
Int : = (R + G + B ) / 3; 

if Int > 0 then 
begin 

Sat 1 - (3 / (R + G + B)) * Minimum (R, G, B) ; 

Hue arccos { ( ( (R-G) + (R-B) ) /2) /sqrt (sqr (R-G) + { (R-B) * (G-B.) ) ) ) /(2*pi); 
if (B / Int) > (G / Int) then 
Hue := 1 - Hue; 

end 

else 

begin 

Sat := 0; 

Hue : = 0; 
end ; 

end; // CalculateHSI 

procedure TTooth . LoadBitmapFromFile ( Filename : String); 
begin 

ToothBitmap . LoadFromFile ( Filename ) ; 
ToothBitmap . Dormant ; 

ToothBitmapMask. Assign (ToothBitmap) ; 
end; // LoadBitmapFromFile 



end. 
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unit RefDisplay; 
interface 



uses 



Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs, 
Grids, ShadeData, ComCtrls, StdCtrls, ExtCtrls; 



type 

TfrmRef erenceDisplay - class (TForm) 
Panell: TPanel; 
sgAnalysis: TStringGrid; 
Panel2: TPanel; 
edGridCol: TEdit; 
udGridCol: TUpDown; 
edGridRow: TEdit; 
udGridRow: TUpDown; 
Labell : TLabel ; 
Label2: TLabel; 
Label3: TLabel; 

procedure FormCreate ( Sender : TObject); 

procedure InsertShadeData (Shade : TShadeColours) ; 

procedure edGridColChange (Sender : TObject) ; 

procedure LoadShades (Shades : TShadeRef erences ) ; 

procedure edGridRowChange ( Sender: TObject); 
private 

{ Private declarations ) 

Rowlnsertlndex : integer; 

DisplayRow : integer; 

DisplayColumn : integer; 

DisplayShades : TShadeRef erences ; 

procedure ShowShades; 
public 

{ Public declarations } 
end; 

var . 

frmRef erenceDisplay: TfrmRef erenceDisplay ; 

implementation 

($R * . DFM} 

const 

TitleRow » 0; 
Name Column = 0; 
RedColumn = NameColumn + 1; 
GreenColumn = RedColumn +1; 
BlueColumn « GreenColumn + 1; 
Variat ionColumn = BlueColumn +1; 
RedMaxColumn = Variat ionColumn + 1; 
RedMinColumn ~ RedMaxColumn + 1; 
GreenMaxColumn « RedMinColumn + 1; 
GreenMinColumn - GreenMaxColumn +1; 
BlueMaxColumn = GreenMinColumn + 1; 
BlueMinColumn = BlueMaxColumn + 1 ; 
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Precision =* 6; 
Digits ~ 4; 

procedure TfrmRef erenceDispiay . FormCreate ( Sender : TObject) ; 

^^Configure the Grid to Display the Reference Set } 
DisplayColumn GridWidth div 2; 
DisplayRow : *= GridHeight div 2; 

udGridCol .Min := 1; 
udGridCol .Max :- GridWidth; 

udGridRow.Min := 1; 
udGridRow.Max :« GridHeight; 

udGridRow. Position DisplayRow; 
udGridCol. Position := DisplayColumn; 

edGridCol .Text := IntToStr ( DisplayColumn) ; 
edGridRow.Text :« IntToStr ( DisplayRow) ; 

sgAnalysis.. ColCount := 11; 
sgAnalysis.RowCount 17; // may change 

Rowlnsertlndex := TitleRow +1; ( 
sgAnalysis. Cells [NameColumn, TitleRow] := 'Shade ; 
sgAnalysis. Cells [RedColumn, TitleRow) := *Red'; 
sgAnalysis. Cells [GreenColumn, TitleRow] := 'Green'; 
sgAnalysis. Cells [BlueColumn, TitleRow) 'Blue'; 
sgAnalysis. Cells [ Variat ionColumn, TitleRow) 'Variation ; 

sgAnalysis. Cells [ RedMaxColumn, TitleRow] 'Max Red'; 

sgAnalysis. Cells [ GreenMaxColumn, TitleRow) := 'Max Green ; 
sgAnalysis. Cells [BlueMaxColumn, TitleRow.) 'Max Blue'; 

sgAnalysis. Cells [ RedMinColumn, TitleRow] := 'Mm Red'; 
sgAnalysis. Cells [GreenMinColumn, TitleRow] :~ 'Mm Green ; 
sgAnalysis. Cells [BlueMinColumn, TitleRow] 'Min Blue'; 

DisplayShades := TShadeRef erences . Create; 
end; 

procedure TfrmRef erenceDispiay. InsertShadeDat a {Shade : TShadeColours ) ; 
var 

Variation : real; 
begin 

with Shade. GridCoiours [DisplayColumn, DisplayRow] do 
begin 

sgAnalysis. Cells [NameColumn, Rowlnsertlndex] := Shade. Name; 

sgAnalysis. Cells [RedColumn,. Rowlnsertlndex] := FloatToStrF(Red, ffFixed, 

PreC igAnalysi^Cells [GreenColumn, Rowlnsertlndex] := FloatToStrF(Green, ffFixed 

PreC sgAnalysis!'ceils [BlueColumn, Rowlnsertlndex] :- FloatToStrF(Blue, ffFixed, 

^"gAnalysirCeilstRedMaxColumn, Rowlnsertlndex] := FloatToStrF ( RedMax , 
ffFixed, Precision, Digits) ; 
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begin 



IniFile . WriteString (IniRef erenceSection, IniRefX, IntToStr (X) ) ; 
IniFile .WriteString ( IniRef erenceSect ion, IniRef Y, IntToStr ( Y) ) ; 
end; 

with SampleLocation do 

^iiiFile .WriteString ( IniSampleSection, IniSampleX, IntToStr (X) ) ; 
IniFile .WriteString < IniSampleSection, IniSampleY, IntToStr <Y) ) ; 

end; 
finally 

IniFile . Free; . 
end; 
end; 

Close; 
end; 

procedure Tf rmSampleLocator .btnLoadSampleClicM Sender : TObject); 

begin , ( 

OpenDialog. Title := 'Sample Imade To Display ; 

OpenDialog. InitialDir :.- Copy ( ParamStr ( 0 ) , 0 , 3) + 'AnalyseVPicturesV ; 
OpenDialog. DefaultExt := GraphicExtension (TBitmap) ; 
OpenDialog. Filter :« GraphicFil ter (TBitmap) ; _ 
OpenDialog. Options :« [of PathMus tExist , of FileMustExist ] ; 
if OpenDialog. Execute then 

begin t ^ _ , . 

pmilmage . Picture . LoadFromFile (OpenDialog . Filename) ; 

end; 
end; 

procedure Tf rmSampleLocator . pmilmageClick { Sender : TObject); 

^^Check Location Option and Update The Location Co Ords ) 
if rgLocation. Itemlndex = 0 then 
begin 

edRefX.Text := edXPos.Text; 

edRefY.Text := edYPos.Text; 
end 
else 
begin 

edSampleX.Text edXPos.Text; 
edSampleY.Text := edYPos.Text; 
end; 
end; 

procedure Tf rmSampleLocator . FormShow (Sender : TObject); 
begin 

edRefX.Text := IntToStr (Ref erenceLocat ion . x) ; 
edRefY.Text := IntToSt r ( Re f erenceLocation . y ) ; 
edSampleX.Text := IntToStr ( SampleLocation . x ) ; 
edSampleY.Text IntToStr ( SampleLocation . y) ; 
end; 

procedure T f rmSampleLocator . btnCancelClick ( Sender : TObject); 
begin 
Close; 
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unit SampleLocator; 
interface 



"Endows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs, 
TMultiP, ExtCtrls, StdCtrls; 

type 

Tf rmSampleLocator = class (TForiu) 
Panell: TPanel; 
pmilmage: TPMult ilmage ; 
OpenDialog: TOpenDialog; 
btnLoadSample : TButton; 
edXPos: TEdit; 
Labell: TLabel; 
edYPos: TEdit; 
Label2: TLabel; 
rgLocation : TRadioGroup; 
Panel2: TPanel; 
Label3 : TLabel; 
Label4 : TLabel; 
edRefX: TEdit; 
LabelS : TLabel; 
edRefY: TEdit; 
Label6: TLabel; 
edSampleX: TEdit; 
Label 7 : TLabel; 
edSampleY: TEdit; 
Label8: TLabel; 
btnSave: TButton; 

procedure * plilmageMouseMovef Sender : TObj ect ; Shift: TShiftState; X, 

Y: Integer) ; 

procedure FormCreate { Sender : TObject); 

procedure btnSaveClick (Sender : TObject); 

procedure btnLoadSampleClick ( Sender : TObject); 

procedure pmilmageClick ( Sender: TObject); 

procedure FormShow {Sender : TObject); 

procedure btnCancelClick ( Sender : TObject); 
private 

{ Private declarations ) 
public 

{ Public declarations ) 

Ref erenceLocation : TPoint; 

SampleLocation : TPoint; 



end; 
var 

f rmSampieLocator : Tf rmSampl eLocat or ; 
implementation 
($R *.DFM} 



SampleLocator - page 1 



WO 00/25696 PCT/US99/22857 

26 



uses 

SDIMain, IniFiles; 



procedure TfrmSampleLocator .pmilmagaMouseMove (Sender = TObject, 
Shift: TShiftState; X, Y: Integer); 



begin 

edXPos.Text IntToStr(X); 
edYPos.Text := IntToStr(Y); 
end; 



procedure Tf rmSample Locator . FormCreate (Sender : TObject) 



var 



IniFile :. TIniFile; 

be ^Load The Saved Sample Location From Ini File } - 
{ Set Default for now } 
Ref erenceLocation := Point (170, 40); 
SampleLocation :=» Point (300, 160); 

. if FileExists(frmShadeAnalyzer.DiskDrive + 'AnalyseV + IniFileName) then 
begin 

tr iniFile TIniFile. Create ( f rmShadeAnalyzer . DiskDrive + -AnalyseV + 

IniFileName) ; 

with Ref erenceLocation do 

StrToIntdniFile.ReadStringdniReferenceSecticn^IniRefX, > > 

Y := StrToInt (IniFile. ReadString ( IniRef erenceSection, IniRef Y, ERROR )) 
end; 

with SampleLocation do 

b T?= StrToInt<IniFile.ReadSt^ 

y ;s * StrToInt (IniFile. ReadString ( IniSampleSectxon, IniSampleY , ERROR )) 

end; 
finally 

IniFile. Free; 
end; 
end; 
end; 

procedure Tf rmSampleLocator . btnSaveClick (Sender : TObject); 
var 

IniFile : TIniFile; 

be ?eferenceLocation := point (StrToInt ( edRefX . Text ) , StrToInt ( edRefY . Text )) ; 
Kmp^oc^ point (StrToInt(edSampleX. Text), StrToInt (edSampleY . Text )> 

if FileExiststfrmShadeAnalyzer.DiskDrive + 'AnalyseV + IniFileName) then 
begin 

tr iniFile TIniFile . Create ( f rmShadeAnalyzer . DiskDrive + 'AnalyseV + 
IniFileName) ; 

with Ref erenceLocation do 
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end; 
end. 
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unit Sdimain; 
interface 

uses Windows, Classes, Graphics, Forms, Controls, Menus, 
Dialogs, StdCtrls, Buttons, ExtCtrls, ComCtrls, 
ShadeData; 

« * ShadeAnalyse . ini ' ; 

= * REFERENCE AREA* ; 
« • RefAreaX•; 
= 'RefAreaY' ; 

« 1 SAMPLE AREA 1 ; 
« 1 SampleAreaX* ; 
« 1 SampleAreaY 1 ; 

= ' DEFAULT GUIDE ' ; 
= ' GuideFilename 1 ; 

= true; // used for splash screen 

type .... 
TfrmShadeAnalyzer = class (TForm) 
SDIAppMenu: TMainMenu; 
FileMenu: TMenuItem; 
Exit Item: TMenuItem; 
Nl : TMenuItem; 
OpenDialog: TOpenDialog; 
Helpl: TMenuItem; 
Aboutl: TMenuItem; 
StatusBar: TStatusBar; 
Calibrate: TMenuItem; 
Optionsl : TMenuItem; 
Showlmagel: TMenuItem; 
ShowRef erencel : TMenuItem; 
SetSampleLocl : TMenuItem; 
Analysel: TMenuItem; 

gbShadeSet: TGroupBox; 
btnLoad: TButton; 

btnSave:' TButton; 

edShadeSetName : TEdit; 

gbSampleAnalysis: TGroupBox; 

btnMatch: TButton; 

Labell: TLabel; 

edNearest: TEdit; 

SaveDialog: TSaveDialog; 

procedure ShowHint (Sender : TObject); 

procedure About lClick ( Sender : TObject); 

procedure FormCreate (Sender : TObject); 

procedure CalibrateClick (Sender: TObject); . 

procedure ShowImagelClick ( Sender : TObject); 

procedure ShowRef erencelClick (Sender : TObject); 

procedure SetSampleLoclClick ( Sender : TObject) ; 

procedure AnalyselClick ( Sender : TObject); 



const 

IniFilename 

IniRef erenceSection 

IniRefX 

IniRefY 

IniSampleSection 

IniSampleX 

IniSampleY 

. iniShadeSetSection 
IniDef aultGuide 

Startup : Boolean 
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procedure btnSaveClick (Sender : TObject) ; 
procedure btnLoadClick (Sender : TObject); 

procedure FormClose ( Sender : TObject; var Action: TCloseAct ion) ; 
procedure FormActivate (Sender : TObject); 
private 

{ Private declarations } 
Shades : TShadeRef erences ; 

function AnalyseImage(FileName : string; ShadeName : string) : 
TShadeColours; 

function FindNearestShade (Sample : tShadeColours ) : strings- 
procedure LoadShadeSet (Filename : string); 

public 

{ Public declarations } 
DiskDrive : string; 
NewCalibration : boolean; 

end; 

var 

f rmShadeAnalyzer : Tf rmShadeAnalyzer ; 
implementation 
uses 

SysUtils, About, IniFiles, . 
ToothObject, ImageDisplay, RefDisplay, SampleLocator , SplashScreen; 

($R * . DFM} 
const 

RefRedMedian = 0.5432; 
RefGreenMedian = 0.6308; 
RefBlueMedian « 0.3355; 

Re f Rows = 1; ' 
Ref Columns = 1; 

SampleRows - GridHeight; // To change see Shade Data 

SampleColumns = GridWidth; 

procedure Tf rmShadeAnalyzer . ShowHint (Sender: TObject); 
begin 

StatusBar. SimpleText Application . Hint ; 
end; 

procedure Tf rmShadeAnalyzer . AboutlClick (Sender : TObject); 

begin 

AboutBox . ShowModal ; 
end; 

procedure Tf rmShadeAnalyzer . FormCreate (Sender : TObject); 
var 

IniFile : TIniFile; 
Def aultShadeFilename : strings- 
begin 

Applicat ion.OnHint := ShowHint ; 
DiskDrive := Copy ( ParamSt r ( 0 ) , 0 , 3 ) ; 

Shades TShadeRef erences . Create ; // we will build a new list 
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begin 

if Visible then 
begin 

DisplayRow : = StrToInt ( edGridRow . Text ) ; 
ShowShades; 
. end; 
end; 

end. 
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unit ShadeData; 

{ The Shade reference is divided into a grid. Each area of the grid is 

analysed, and the average Red/ Green and Blue values are stored. If there 
is an edge element in the sample, the valid flag is set to ignore the 
area in correlation matches. 

} 

interface 

uses 

Classes; 

const 

GridWidth - 10; 
GridHeight = 10; 



type 

PShadeColour Element 
TShadeColour Element 



used) 



TShadeColour Element) 
real ) ; 



^TShadeColour Element ; 
class (TObject ) 

Red ; real48; 

Green : real4 8; 

Blue ; rea!48; 

Valid : boolean; 



// Average red content 
// Average green content 
// Average blue content 
// Valid for comparison 



ValidPixelPercent : real48; // 0..1 (1 = all pixels 

RedDev : real48 ; 
GreenDev : real48; 
BlueDev : real48; 
RedMax : real4 8; 
RedMin : . real48; 
GreenMax : real48; 
GreenMin : real48; 
BlueMax : rea!48; 
BlueMin : rea!48; 
constructor Create; 

function ColourDif f erence (ShadeColour : 
real48; 

procedure StoreColour (R, G, B : real4 8; Percent. : 



procedure AddColour ( R, G, B : rea!48; Percent 
function ValidCell : boolean; 

procedure SaveToStream ( OutSt ream : TStream) ; 
procedure LoadFromStream ( InStream :. TStream) 
private 
end; 



real) 



// Name of the Shade reference 



PShadeColours =» ^TShadeColours ; 
TShadeColours = class ( TObj ect ) 

Name : string; 
etc. 

GridColours : array [ 1 GridWidth , 1. . GridHeight ] of 

TShadeColourElement ; 

function Colour Difference ( ShadeColours : TShadeColours). : 

real48; 

procedure SaveToStream (OutStream : TStream); 
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procedure LoadFromStream ( InS tream : TStream); 
private 
end; 

TShadeReferences = class (TObject) 

ShadeList : TList; 
constructor Create; 

procedure AddSample (Sample : TShadeColours) ; 
procedure Clear; 
procedure SortList; 
procedure ReduceList; 

procedure SaveToStream (OutStream : TStream); 
procedure LoadFromStream (InStream : TStream); 
private 
end; 

implementation 
uses 

SysUtils, Dialogs, Controls; 
C °vlJidityLimit -6.95; // 95% of pixels must be used 

constructor TShadeColourElement .Create; :jv 
begin 

Red :« 0; 

Green := 0; 

Blue :» 0; 

ValidPixelPercent :« 0; 
Valid :« false; 
RedDev 0; 
GreenDev : «= 0; 
BlueDev :« 0; 
RedMax :* 0; 
RedMin :«= 1000; 
GreenMax := 0; 
GreenMin :« 1000; 
BlueMax := 0; 
BlueMin :« 1000; 
end; 

procedure TShadeColourElement . StoreColour (R, G, B : real48; Percent : real); 

begin 

Red := R; 
Green G; 
Blue := B; 

ValidPixelPercent := Percent; ■ , 

Valid := (Percent >*= Validi tyLimit ) ; 
end; . 

procedure TShadeColourElement . AddColour ( R, G, B : real48; Percent : real); 
begin 

if R > RedMax then RedMax R; 

if G > GreenMax then GreenMax := G; 

if B > BlueMax then BlueMax : - B ; ^ 
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if R < RedMin then RedMin := R; 
if G < GreenMin then GreenMin :« G; 
if B < BlueMin then BlueMin := B; 
Red := Red + R.- 
Green := Green + G; 
Blue := Blue + B; 
end; 

function TShadeColourElement . ValidCell : boolean; 

begin 

Result := Valid; 
end; 

function TShadeColourElement.ColourDifference(ShadeColour : TShadeColourElement ) 

: real48; 
var 

DistanceRed : real48; 
DistanceGreen : real48; 
DistanceBlue : real48; 
begin 

if (Valid) and ( ShadeColour . Valid) then 
begin 

DistanceRed: = (Red - ShadeColour . Red) ; 
. DistanceGreen:=(Green - ShadeColour . Green) ; 
DistanceBlue:- (Blue - ShadeColour . Blue ) ; 
Result := sqrt (sqr (DistanceRed) + . 

sqr ( DistanceGreen) + 

sqr (DistanceBlue) ) ; 

end- 

else . . . 

Result -1; // cannot compare if any element is invalid 

end; 

procedure TShadeColourElement . SaveToStream (OutSt ream : TStream); 
begin 

OutStream. WriteBuf fer (Red, SizeOf ( Red) ) ; 
OutStream. WriteBuf f er (Green, SizeOf (Green) ) ; 
OutStream.WriteBuf fer (Blue, SizeOf ( Blue )) ; 
OutStream. WriteBuf fer (Valid, SizeOf (Valid) ) ; 

OutStream. WriteBuf fer ( ValidPixelPercent , SizeOf (ValidPixelPercent ) ) ; 
OutStream. WriteBuf fer (RedDev, SizeOf (RedDev) ) ; 
OutStream. WriteBuf fer (GreenDev, SizeOf (GreenDev) ) ; 
OutStream. WriteBuf fer (BlueDev, SizeOf (BlueDev) ) ; 
OutStream. WriteBuf fer (RedMax, SizeOf (RedMax) ) ; 
OutStream. WriteBuf fer (RedMin, SizeOf (RedMin) ) ; 
OutStream. WriteBuf fer (GreenMax, SizeOf (GreenMax) ) ; 
OutStream. WriteBuf fer (GreenMin, SizeOf ( GreenMin) ) ; 
OutStream. WriteBuf fer (BlueMax, SizeOf ( BlueMax ) ) ; 
OutStream. WriteBuf fer (BlueMin, SizeOf (BlueMin) ) ; 
end; 

procedure TShadeColourElement .LoadFromStream < InStream : TStream); 
begin 

InStream. ReadBuf fer (Red, S i zeOf (Red) ) ; 
InStream. ReadBuf fer (Green, SizeOf (Green) ) ; 
InStream. ReadBuf fer (Blue, SizeOf (Blue) ) ; 
InStream. ReadBuf fer (Valid, SizeOf (Valid) ) ; 
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InStream. ReadBuf fer (ValidPixelPercent, SizeOf ( ValidPixelPercent ) ) ; 
InStream. ReadBuf fer(RedDev, SizeOf (RedDev) ) ; 
InStream.ReadBuf fer (GreenDev, SizeOf ( GreenDev) ) ; 
InStream. ReadBuf fer (BlueDev, SizeOf ( BlueDev) ) ; 

InStream.ReadBuf fer (RedMax, SizeOf (RedMax) ) ; ; 
InStream.ReadBuf fer (RedMin, SizeOf (RedMin) ) ; - 
InStream. ReadBuf f er (GreenMax, SizeOf (GreenMax) ) ; 
InStream. ReadBuf fer (GreenMin, SizeOf (GreenMin) ); 
InStream. ReadBuf fer { BlueMax, Si zeOf ( BlueMax) ) ; 
InStream.ReadBuf fer (BlueMin, SizeOf { BlueMin )) ; 
end; 

function TShadeColours.ColourDifference(ShadeColours : TShadeColours ) : real48 

^DifferenceGrid : array [ 1 . . GridWidth, 1. . GridHeight ] of real48; 

Widthlndex : integer; 

Heightlndex : integer; 

MatchedCells : integers- 
begin 

Result := 0; 

MatchedCells 0; 

{ Compare each grid positions colour } 
for Widthlndex 1 to GridWidth do 

for Heightlndex := 1 to GridHeight do 

DifferenceGrid [Widthlndex, Heightlndex] GridColours [Widthlndex, 
Heightlndex] . ColourDiff erence (ShadeColours . GridColours [Widthlndex, 
Heightlndex] ) ; 

end; 

( Calculate the Colour Difference for the whole Shade 
initially just sum the differences 

Possibly just return the standard deviation or something. 

) 

for Widthlndex := 1 to GridWidth do 

for Heightlndex :« 1 to GridHeight do 

if Dif ferenceGrid[WidthIndex, Heightlndex] <> -1 then 

^Result := Result + Sqr ( Dif ferenceGrid [Widthlndex, Heightlndex]); 

inc (MatchedCells) ; 
end; 

Result := Sqrt ( Result /MatchedCells ) ; 
end; 

procedure TShadeColours . SaveToStream (OutSt ream : TStream) ; 
var 

Widthlndex : integer; 
Heightlndex : integer; 
StringLength : integers- 
begin 

StringLength := Length (Name ) ; 

OutStream.WriteBuf fer (StringLength, SizeOf (StringLength) ) ; 
OutStream.WriteBuf fer (Name [1] , StringLength) ; 
for Widthlndex := 1 to GridWidth do 
"for Heightlndex : =* 1 to GridHeight do 
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GridColours [ Widthlndex, Height Index ] . SaveToStream (OutStream) ; 

end; 

procedure TShadeColours . LoadFromStream ( InS t ream : TStream) ; 
var 

Widthlndex : integer; 
Heightlndex : integer; 
StringLength : integer; 

^InStream. ReadBuf f er (StringLength, SizeOf (StringLength).) ; 
SetLength (Name, StringLength); 
InStream. ReadBuf fer (Name [1] , StringLength) ; 
for Widthlndex :- 1 to GridWidth do 

for Heightlndex :- 1 to GridHeight do 

begin 

GridColours (Widthlndex, Heightlndex] : = TShadeColourElement . Create ; 
GridColours (Widthlndex, Heightlndex] . LoadFromStream ( InStream) ; 
end; 

end; 

procedure TShadeRef erences .Adds ample ( Sample : TShadeColours); 
begin 

ShadeList .Add(Sample) ; 
end; 

procedure TShadeRef erences . Clear ; 
begin 

ShadeList .Clear; 
end; 

constructor TShadeRef erences . Create ; 
begin 

ShadeList := TList . Create; 
end; . - 

function SortCompare (Iteml, Item2: pointer).: Integer; 
begin 

if TShadeColours (Iteml) .Name < TShadeColours ( I tem2 ). Name then 
Result : = -1 

else if TShadeColours (Iteml) .Name > TShadeColours ( I tem2 ). Name then 

Result := 1 - . 

else 

Result 0; 

end; 

procedure TShadeRef erences . SortList ; 
begin 

ShadeList .Sort (SortCompare) ; 
end; 

procedure TShadeRe f erences . ReduceList ; 
var 

AverageShadeColours : TShadeColours ; 
AveragedShades : TList; 
CurrentShade : TShadeColours; 
Shadelhdex : integer; 
Row, Col : integer; 
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AverageCount : array [ 1 .. GridWidth, 1 . . GridHeight ] of integer; 
b TFor each individually Named shade, average all values into one ShadeColours 
} 

// New (AverageShadeColours) ; 

AveragedShades := TList . Create; 
//New (CurrentShade) ; 

CurrentShade : ~ TShadeColours . Create; 
CurrentShade .Name : = ' '; 

for Shadelndex := 1 to ShadeList . Count do 

bG if "cur rentShade. Name <> TShadeColours (ShadeList . I terns [ Shadelndex-l ] ) .Name 
then 

begin 

if Shadelndex <> 1 then 
begin 

{ Save the last shade and start a new one } 
for Col := 1 to GridWidth do 
for Row :«= 1 to GridHeight do 

if AverageCount [Col, Row] <> 0 then 

with AverageShadeColours. GridColours [Col, Row) do 

begin 

Red := Red / AverageCount [Col , Row]; 
Green Green / AverageCount (Col , Row] ; 
Blue := Blue / AverageCount [Col , Row] ; 

ValidPixelPercent := ValidPixelPercent / AverageCount [Col , Row] 
end; 

AveragedShades. Add (AverageShadeColours) ; 
end; 

{This is a new Shade} 

AverageShadeColours := TShadeColours . Create ; 
for Col := 1 to GridWidth do 

for Row := 1 to GridHeight do 

begin 

AverageShadeColours .GridColours [Col, Row] : = 

TShadeColourElement . Create ; 

AverageCount [Col, Row] :» 0; 

AverageShadeColours. Name :== TShadeColours (ShadeList . Items [ Shadelndex- 
1 ] ) . Name ; 
end; 

CurrentShade := ShadeList . I terns [ Shade Index-1 ] ; 

for Col := 1 to GridWidth do 
for Row := 1 to GridHeight do 
begin 

with AverageShadeColours. GridColours [Col, Row] do 
begin 

if CurrentShade. GridColours [Col, RowJ Valid then 
begin 

Valid : = true;' 
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AddColour (CurrentShade ..GridColours [Col , Row] .Red, 

CurrentShade.GridColours [Col, Row]. Green, 
CurrentShade. GridColours [Col, Row]. Blue, 
CurrentShade.GridColours [Col, Row] .ValidPixelPercent) ; 

N ValidPixelPercent := ValidPixelPercent + 

CurrentShade.GridColours [Col, Row] .ValidPixelPercent; 



// 
'// 
// 



Red := Red + CurrentShade . GridColours [Col , Row] . Red; 
Green := Green + CurrentShade . GridColours [Col , Row]. Green; 
Blue Blue + Current Shade . GridColours (Col , Row]. Blue; 
inc (AverageCount [Col, Row]); 
end; 
end; 



end; 

end; 

{ Save the last shade } 
for Col := 1 to GridWidth do 
for Row := 1 to GridHeight do 

if AverageCount [Col, Row] <> 0 then 

with AverageShadeColours. GridColours [Col, Row] do 

begin 

Red Red / AverageCount [Col , Row]; 
Green := Green / AverageCount [Col , Row]; 
Blue := Blue / AverageCount [Col , Row]; 

ValidPixelPercent := ValidPixelPercent / AverageCount [Col, Row]; 
end; 

AveragedShades .Add (AverageShadeColours) ; 

ShadeList . Free; 
ShadeList : - AveragedShades; 
end; 

procedure TShadeRef erences . SaveToSt ream (OutStream :.TStream); 
var 

Shadelndex : integer; 
CurrentShade : TShadeColours ; 

^Shadelndex ':- ShadeList . Count ; // First write the number of Shade in the s 
OutStream. WriteBuf f er (Shadelndex, SizeOf (Shadelndex) ) ; 
for Shadelndex := 0 to ShadeList . Count - 1 do 

begin ' 

CurrentShade : = ShadeList . I terns [Shadelndex] ; 

CurrentShade . SaveToSt ream (OutStream) ; 
end; 
end; 

procedure TShadeRef erences . LoadFromSt ream ( InStream : TStream) ; 
var 

NumberOf Shades : integer; 
Shadelndex : integer; 
CurrentShade, : TShadeColours; 
begin 

InStream. ReadBuff er ( NumberOf Shades , SizeOf (NumberOf Shades ) ) ; 

for Shadelndex : - 0 to NumberOf Shades - 1 do 

begin 

CurrentShade := TShadeColours . Create ; 
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CurrentShade . LoadFromStream ( InStream) 
AddSample (CurrentShade) ; 
end; 
end; 

end . 
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